Buy Sell Discuss UK Domain Names at AcornDomains.co.uk

Today's drop dates are 01-06-2008 & 08-06-2008  
Domain Name Sales Domain Software Cheap Domain Registration Domain Lists SedoPro Code Domains For Sale NameDrive Domain Parking
Go Back   Domain Name Forum Acorn Domains Buy Sell Trade UK Domain Names > Website Design and Promotion > Search Engine Optimisation

Search Engine Optimisation Discuss Search Engine Optimisation methods. How to assess traffic on a domain name and build traffic on your portfolio.

Reply
 
Thread Tools Display Modes
Old 26-12-2006, 04:31 PM   #1
Administrator
 
admin's Avatar
 
Join Date: Jun 2004
Posts: 6,013
admin has disabled reputation

Question Yahoo Slurp

Yahoo loves this forum, there are currently 35 spiders on it right now crawling the site.

Anyone know how to calm this down?

It's using over 1GB a month in bandwidth.

Thanks

Admin
__________________
9p Domains | Domain Appraisal | Droplists | Domain Brokerage | Find Valuable Domains | Link to us | Domain Software
New to UK Domain Names? - Go Here | Check Drop Dates | Domain Transfers | Game Consoles
Make more money from your Domain Names with SedoPro - PM for a free activation code!
admin is offline   Reply With Quote
Old 26-12-2006, 07:19 PM   #2

 
stender's Avatar
 
Join Date: Nov 2005
Posts: 954
stender has a reputation beyond reputestender has a reputation beyond reputestender has a reputation beyond reputestender has a reputation beyond reputestender has a reputation beyond reputestender has a reputation beyond reputestender has a reputation beyond reputestender has a reputation beyond reputestender has a reputation beyond reputestender has a reputation beyond reputestender has a reputation beyond repute

Not sure of your os or if this helps but....

Clockwatchers Web Hosting - robots.txt Tutorial - Block Bad Bots
stender is offline   Reply With Quote
Old 27-12-2006, 10:55 AM   #3

 
Join Date: Jun 2005
Location: Manchester
Posts: 445
stevie has a reputation beyond reputestevie has a reputation beyond reputestevie has a reputation beyond reputestevie has a reputation beyond reputestevie has a reputation beyond reputestevie has a reputation beyond reputestevie has a reputation beyond reputestevie has a reputation beyond reputestevie has a reputation beyond reputestevie has a reputation beyond reputestevie has a reputation beyond repute

you could try a crawl delay for slurp

User-agent: Slurp
Crawl-delay: 10


Yahoo! Help -
stevie is offline   Reply With Quote
Reply



Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off
Forum Jump

Similar Threads
Thread Thread Starter Domain Name Community Replies Last Post
OpenDomain Donates Use of Web Addresses to Encourage Online ... - Yahoo! News (press release) RSS Domain Name News 0 29-09-2006 04:59 PM
Yahoo Status Page LeeOwen General Board 1 22-07-2006 10:37 PM
dotMobi Announces Extension of Trademark Registration - Yahoo! News (press release) RSS Domain Name News 0 10-07-2006 03:59 PM
Microsoft Office Live catching up with Yahoo! - iTWire RSS Domain Name News 0 31-05-2006 04:00 AM
Yahoo Site Explorer admin Search Engine Optimisation 4 12-02-2006 05:30 AM


All times are GMT +1. The time now is 02:10 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81