datascraping tools

[stevieboy1101]

Hi

does anyone know of any datascraping tools thats takes product data from a product page and can store in a database. Needs to be linuxed based and needs to automate around a cron job?

Thanks
Steve.

 

[Heavy]

Not off the shelf. How much data and what's the complexity of the product data?

You've got http://nutch.apache.org/ depending what you're doing it may be overkill. If I'm scraping simple stuff I've created a few rough custom scripts before eg scraping property websites to alert me when something interesting comes up matching given criteria. Drop me a PM if you fancy

 

[wonder_lander]

If the website has an affiliate program their is often a datafeed provided so no need to scrape.