datascraping tools

[stevieboy1101]

Hi

does anyone know of any datascraping tools thats takes product data from a product page and can store in a database. Needs to be linuxed based and needs to automate around a cron job?

Thanks
Steve.

 

[Heavy]

Not off the shelf. How much data and what's the complexity of the product data?

You've got http://nutch.apache.org/ depending what you're doing it may be overkill. If I'm scraping simple stuff I've created a few rough custom scripts before eg scraping property websites to alert me when something interesting comes up matching given criteria. Drop me a PM if you fancy

 

[seemly]

http://import.io/

Turn any website into a table of data or an API in minutes without writing any code

We provide a scalable solution that lets you create, manipulate and explore Data Sets.

Use them to inform business decisions, feed data visualizations, and power apps.

 

[wonder_lander]

If the website has an affiliate program their is often a datafeed provided so no need to scrape.