Membership is FREE – with unlimited access to all features, tools, and discussions. Premium accounts get benefits like banner ads and newsletter exposure. ✅ Signature links are now free for all. Share your own thoughts and experience, accounts may be terminated for violations.
  • Welcome to AcornDomains.co.uk, a friendly community for UK domain investors, developers and anyone interested in domain names.

    Whether you're looking to buy or sell domains, learn more about the UK domain market, get feedback on your names, or simply chat with other domainers, you're in the right place. AcornDomains has been bringing together people from across the UK domain industry for many years, from complete beginners to experienced investors.

    Have a look around, join the discussions and, if you haven't already, create a free account and introduce yourself.

wordpress robots.txt

Status
Not open for further replies.
Joined
Nov 8, 2005
Posts
2,499
Reaction score
35
Has anyone a preffered robots.txt for wordpress? I've googled various opinions but wondered if anyone has a proven one they use with a well ranked wp site

found this one

Disallow: /wp-content/
Disallow: /trackback/
Disallow: /wp-admin/
Disallow: /feed/
Disallow: /archives/
Disallow: /sitemap.xml
Disallow: /index.php
Disallow: /*?
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: */feed/
Disallow: */trackback/
Disallow: /page/
Disallow: /tag/
Disallow: /category/

User-agent: Googlebot-Image
Disallow: /wp-includes/

User-agent: Mediapartners-Google*
Disallow:

User-agent: ia_archiver
Disallow: /

User-agent: duggmirror
Disallow: /
 
Last edited:
Anyone? I have a site ready for promotion just gotta sort the robots and terms/privacy pages and i can start promotion.
 
Hmm doesn't look so good that one...

Why is it disallowing the sitemap, and why does it have no sitemap: rule?

and why isn't it allowing google images hmm...

I would just grab one off the bigger wp blogs like techcrunch or something.
 
Not sure really, I've found so many different examples and one telling you one thing.......also found a robots plugin which is equally confusing.
I looked at tech crunches and not sure about that. they don't show any wp directories.
 
Status
Not open for further replies.
Top Bottom