Domain Manage

Blocking Baidu

Discussion in 'Website Design' started by Sussexite, Dec 20, 2011.

Thread Status:
Not open for further replies.
  1. Sussexite

    Sussexite Active Member

    Joined:
    Oct 2010
    Posts:
    278
    Likes Received:
    8
    Has anyone found an effective way of blocking the Baidu spider.

    It ignores robots.txt by all accounts and seems to be ignoring my attempts to block it using .htaccess

    Perhaps I'm coding .htaccess wrong.

    I've used

    order allow,deny
    deny from baidu.com
    deny from crawl.baidu.com
    deny from 220.181.7.
    deny from 123.125.66.
    deny from 180.76.5.
    deny from 93.158.
    allow from all

    Any help gratefully received
     
  2. Domain Forum

    Acorn Domains Elite Member

    Joined:
    1999
    Messages:
    Many
    Likes Received:
    Lots
     
  3. admin Spain

    admin Administrator Staff Member

    Joined:
    Jun 2004
    Posts:
    10,084
    Likes Received:
    115
    From what I know, Baidu uses 119.63.192.0 - 119.63.199.255.

    It's better not to block anything but spammers by IP, so try this instead:

    RewriteCond %{HTTP_USER_AGENT} Baiduspider
    RewriteRule ^.*$ http://127.0.0.1 [R,L]

    Admin
     
  4. Sussexite

    Sussexite Active Member

    Joined:
    Oct 2010
    Posts:
    278
    Likes Received:
    8
    Thanks Admin

    I tried that yesterday, but to no avail.

    Does it matter where in the .htaccess file this code is placed eg before or after the Begin Wordpress code etc?

    Sussexite
     
  5. fish United Kingdom

    fish Well-Known Member

    Joined:
    Nov 2006
    Posts:
    2,725
    Likes Received:
    27
    I use this site to generate code to block either by IP or site.

    Seems to work for me, I'm building a lovely black list of websites/ips that I add to all of mine via htaccess :)
     
  6. Sussexite

    Sussexite Active Member

    Joined:
    Oct 2010
    Posts:
    278
    Likes Received:
    8
    Very neat Fish, thanks
     
Thread Status:
Not open for further replies.

Share This Page