20i Reseller Hosting

datascraping tools

Discussion in 'Content Management Systems' started by stevieboy1101, Dec 7, 2013.

Thread Status:
Not open for further replies.
  1. stevieboy1101 United Kingdom

    stevieboy1101 Active Member

    Joined:
    Mar 2011
    Posts:
    65
    Likes Received:
    0
    Hi

    does anyone know of any datascraping tools thats takes product data from a product page and can store in a database. Needs to be linuxed based and needs to automate around a cron job?

    Thanks
    Steve.
     
  2. Domain Forum

    Acorn Domains Elite Member

    Joined:
    1999
    Messages:
    Many
    Likes Received:
    Lots
    articles.co.uk
     
  3. Heavy Australia

    Heavy Member

    Joined:
    Dec 2012
    Posts:
    5
    Likes Received:
    0
    Not off the shelf. How much data and what's the complexity of the product data?

    You've got http://nutch.apache.org/ depending what you're doing it may be overkill. If I'm scraping simple stuff I've created a few rough custom scripts before eg scraping property websites to alert me when something interesting comes up matching given criteria. Drop me a PM if you fancy
     
  4. seemly

    seemly Well-Known Member

    Joined:
    Feb 2011
    Posts:
    1,263
    Likes Received:
    208
    http://import.io/

     
  5. wonder_lander United Kingdom

    wonder_lander Well-Known Member Full Member

    Joined:
    Mar 2009
    Posts:
    1,019
    Likes Received:
    89
    If the website has an affiliate program their is often a datafeed provided so no need to scrape.
     
Thread Status:
Not open for further replies.