Domain Forum Acorn Domains Buy Sell Auction UK Domains - View Single Post - Does your site get crawled by Google if you URL is found on another page it has crawl

admin · 04-10-2005, 06:06:04 PM

September Google Crawler Stats:

Googlebot 4507+75 110.46 MB 30 Sep 2005 - 22:01

Anyone know what this really means? Taken from Advanced Web Statistics 6.4

We have put controls in our HTaccess file to stop Google crawling Pms and opening too many connections as we found at times the Googlebot had 20 connections open at once when spidering the site.

04-10-2005, 06:06:04 PM	#5 (permalink)
admin Administrator Join Date: Jun 2004 Posts: 8,601 No Classified Ratings Domain Trader Rating: (100% / 1)	September Google Crawler Stats: Googlebot 4507+75 110.46 MB 30 Sep 2005 - 22:01 Anyone know what this really means? Taken from Advanced Web Statistics 6.4 We have put controls in our HTaccess file to stop Google crawling Pms and opening too many connections as we found at times the Googlebot had 20 connections open at once when spidering the site. __________________ Domain Appraisal \| Domain Scripts \| Domain Software\| link to Acorn Domains