Membership is FREE – with unlimited access to all features, tools, and discussions. Premium accounts get benefits like banner ads and newsletter exposure. ✅ Signature links are now free for all. 🚫 No AI-generated (LLM) posts allowed. Share your own thoughts and experience — accounts may be terminated for violations.

Good robots.txt for SEO?

Status
Not open for further replies.
Joined
Aug 12, 2004
Posts
251
Reaction score
5
I've currently got a holding page up for one of my domains. It's on google through old back links thus the googlebot amongst others is crawling it.

Does anyone have an example of a good robots.txt file that I should put up that is search engine friendly. Or does it no make a difference?
 
None is fine.

However if you've got a large site with large quantities of information it can be very expensive in terms of bandwidth to keep feeding the robots.

http://en.wikipedia.org/robots.txt is a good starting point for weeding out the useless robots.
 
Status
Not open for further replies.

Rule #1: Be Respectful

Do not insult any other member. Be polite and do business. Thank you!

Premium Members

New Threads

Domain Forum Friends

Lastest Listings

Our Mods' Businesses

*the exceptional businesses of our esteemed moderators
General chit-chat
Help Users
  • No one is chatting at the moment.
  • Siusaidh AcornBot:
    Siusaidh has left the room.
      Siusaidh AcornBot: Siusaidh has left the room.
      Top Bottom