Blocking Baidu

Sussexite · 20-12-2011, 02:43:43 PM

Has anyone found an effective way of blocking the Baidu spider.

It ignores robots.txt by all accounts and seems to be ignoring my attempts to block it using .htaccess

Perhaps I'm coding .htaccess wrong.

I've used

order allow,deny
deny from baidu.com
deny from crawl.baidu.com
deny from 220.181.7.
deny from 123.125.66.
deny from 180.76.5.
deny from 93.158.
allow from all

Any help gratefully received

admin · 20-12-2011, 03:52:17 PM

From what I know, Baidu uses 119.63.192.0 - 119.63.199.255.

It's better not to block anything but spammers by IP, so try this instead:

RewriteCond %{HTTP_USER_AGENT} Baiduspider
RewriteRule ^.*$ http://127.0.0.1 [R,L]

Admin

Sussexite · 20-12-2011, 04:19:27 PM

Thanks Admin

I tried that yesterday, but to no avail.

Does it matter where in the .htaccess file this code is placed eg before or after the Begin Wordpress code etc?

Sussexite

fish · 20-12-2011, 05:23:40 PM

I use this site to generate code to block either by IP or site.

Seems to work for me, I'm building a lovely black list of websites/ips that I add to all of mine via htaccess

Sussexite · 20-12-2011, 05:49:43 PM

Very neat Fish, thanks

20-12-2011, 02:43:43 PM	#1 (permalink)
Sussexite Join Date: Oct 2010 Location: Sussex Posts: 203 No Classified Ratings Domain Trader Rating: (100% / 2)	Blocking Baidu Has anyone found an effective way of blocking the Baidu spider. It ignores robots.txt by all accounts and seems to be ignoring my attempts to block it using .htaccess Perhaps I'm coding .htaccess wrong. I've used order allow,deny deny from baidu.com deny from crawl.baidu.com deny from 220.181.7. deny from 123.125.66. deny from 180.76.5. deny from 93.158. allow from all Any help gratefully received

20-12-2011, 03:52:17 PM	#2 (permalink)
admin Administrator Join Date: Jun 2004 Posts: 8,601 No Classified Ratings Domain Trader Rating: (100% / 1)	From what I know, Baidu uses 119.63.192.0 - 119.63.199.255. It's better not to block anything but spammers by IP, so try this instead: RewriteCond %{HTTP_USER_AGENT} Baiduspider RewriteRule ^.*$ http://127.0.0.1 [R,L] Admin __________________ Domain Appraisal \| Domain Scripts \| Domain Software\| link to Acorn Domains

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Domain Name Community	Replies	Last Post
Chinese search engine Baidu sues US company over hacking attack - Telegraph.co.uk	RSS	Domain Name News	0	22-01-2010 09:59:12 AM
Baidu Blames Domain Registrar for Security Breach Register.com sued over ... - Softpe	RSS	Domain Name News	0	21-01-2010 10:59:03 AM
Baidu hacked by 'Iranian cyber army' - BBC News	RSS	Domain Name News	0	12-01-2010 11:59:01 AM
Euro Baidu to auction name on eBay - Baltimore Sun	RSS	Domain Name News	0	09-12-2009 05:59:17 PM
Baidu is beginning its international expansion with the purchase ... - PR-Inside.com	RSS	Domain Name News	0	26-09-2009 09:59:02 AM

20-12-2011, 04:19:27 PM	#3 (permalink)
Sussexite Join Date: Oct 2010 Location: Sussex Posts: 203 No Classified Ratings Domain Trader Rating: (100% / 2)	Thanks Admin I tried that yesterday, but to no avail. Does it matter where in the .htaccess file this code is placed eg before or after the Begin Wordpress code etc? Sussexite

20-12-2011, 05:23:40 PM	#4 (permalink)
fish Join Date: Nov 2006 Posts: 2,316 No Classified Ratings Domain Trader Rating: (100% / 46)	I use this site to generate code to block either by IP or site. Seems to work for me, I'm building a lovely black list of websites/ips that I add to all of mine via htaccess

20-12-2011, 05:49:43 PM	#5 (permalink)
Sussexite Join Date: Oct 2010 Location: Sussex Posts: 203 No Classified Ratings Domain Trader Rating: (100% / 2)	Very neat Fish, thanks