Domain Manage

Assistance with PHP DAC script required!

Discussion in 'Domain Tools' started by timter51, Aug 9, 2013.

Thread Status:
Not open for further replies.
  1. timter51 United Kingdom

    timter51 Active Member Exclusive Member

    Joined:
    Oct 2012
    Posts:
    549
    Likes Received:
    22
    I've knocked together a PHP script which I'm using basically as a DAC checker. I'm opening up an fsockopen on the DAC, sending in an array of domains with fwrite, and outputting the returned strings using fgets (whilst using feof to check for end of file). I'm checking 3 domains a second to stay within limits.

    The problem I'm having is that the script exits at random times for no reason, no matter how many domains are in the array. I could get 2000 domains into an array of 10000, and it'll just exit out. I can't work out what's going on... it's like the response from the DAC is cut short.

    Can anyone with experience here take a quick look for me and lend a hand? It'll probably take a someone who knows what they're doing seconds to see where I'm going wrong! Please PM if you might be able to help.
     
  2. Domain Forum

    Acorn Domains Elite Member

    Joined:
    1999
    Messages:
    Many
    Likes Received:
    Lots
     
  3. grantw United Kingdom

    grantw Well-Known Member

    Joined:
    Mar 2005
    Posts:
    4,649
    Likes Received:
    82
    Are you hammering it and getting blocked perhaps causing the script to timeout?

    Sorry, just reread your post, not that then!

    Grant
     
  4. Skinner

    Skinner Well-Known Member

    Joined:
    Jul 2008
    Posts:
    4,325
    Likes Received:
    81
    I have noticed there is sometimes a discrepency with what the Dac Returns, I can't figure out where it comes from but randomly the dac returns what I can only describe a foreign response i.e. something unexpected. With my first scripts it resulted in an infinite loop, I solved it by only acting on N/Y/B.

    The other thing to be aware of running out of is resources if you are limited ? PHP is a nightmare for overheads (memory wise) when dealing with large arrays.

    Next one would be execution time. I set my execution time out to 600 seconds so when loading names I limit it to 14 * 60 * 600 names, could you be timing out ?
     
  5. monaghan United Kingdom

    monaghan Moderator Staff Member

    Joined:
    May 2007
    Posts:
    1,993
    Likes Received:
    45
    If you are not looking for out and out speed, why have such a huge array? Wouldn't you be better off putting the names in a database table and pulling a name, stuff it through the DAC, update the database, pause and then loop? You can use your select statement to just pull the unprocessed records making it easy to restart if you crash out.

    Also as has been suggested, if you are running in a hosted environment then you may well have timeout issues due to PHP settings.
     
  6. Aaron Clifford United Kingdom

    Aaron Clifford Well-Known Member

    Joined:
    Jun 2010
    Posts:
    2,236
    Likes Received:
    29
    Code:
    set_time_limit(0);
    Stick that at the top of your PHP file, it may stop the timing out.

    Also best to make sure that you are sleeping correctly, what code are you using to sleep with?
     
  7. timter51 United Kingdom

    timter51 Active Member Exclusive Member

    Joined:
    Oct 2012
    Posts:
    549
    Likes Received:
    22
    Thanks Skinner, I've not noticed any dodgy responses yet but I'll look into that. You could right on overheads and execution time, though... I'm running this on a local WAMP installation, so I'll see how I have it configured for that.

    Love the database idea, but my PHP skills are limited! It is something I want to get to eventually, I know this is how any serious domainer should be doing it. In the meantime, I have a desperate need for something that just works at my skill level.

    Yes I've got that set_time_limit in, that was my first "aaah, I know what it is" fix, but no luck. I'm using usleep(400000); to sleep each iteration through the array, that seems to work fine as I'm not getting any blocks from the DAC.
     
  8. Skinner

    Skinner Well-Known Member

    Joined:
    Jul 2008
    Posts:
    4,325
    Likes Received:
    81
    I use a triple database system, I'm sure there are better ways but my system was never meant to be this big was just a small personal database, so kinda stuck with it.

    Database 1 is a hopper loaded with keywords and strings, basically its

    id, keyword, scanned, date

    Once I'm done catching for the day I load 600 seconds (5 x 600) worth of names where scanned = 0, running at 200ms intervals (roughly 5x per second, I chose this as it doesn't really mess up my rolling 24 hour totals, but can go as low as 70ms) on a 10 min cron.

    I then separate the chaff from the wheat into 2 databases

    Database 2 is a registered name database
    Database 3 is available names.

    With each iteration, I build an SQL query, and every 100th iteration I update the scanned database of those 100 names to scanned=1. I also run the mysql update when blocked or the loop kicks out.

    I maintain the available names list to get idea's for domains and find useful FTR names too.
     
  9. monaghan United Kingdom

    monaghan Moderator Staff Member

    Joined:
    May 2007
    Posts:
    1,993
    Likes Received:
    45
    Seems odd that as a self declared PHP noob that you find realtime network IO easier than databases, especially given that there is so much database documentation and very little network IO documentation :)
     
  10. Aaron Clifford United Kingdom

    Aaron Clifford Well-Known Member

    Joined:
    Jun 2010
    Posts:
    2,236
    Likes Received:
    29
    ^ That! The database stuff is easy, all you probably need to get going is:

    http://www.w3schools.com/php/php_mysql_select.asp
    http://www.w3schools.com/php/php_mysql_update.asp

    Then you could even use something like:

    http://sanderkorvemaker.nl/test/flexigrid/

    to view the database, or just use phpmyadmin.
     
  11. PaulGregory

    PaulGregory Active Member

    Joined:
    Jun 2013
    Posts:
    230
    Likes Received:
    2
    Thing is with DBs there are multiple ways of reaching the same goal, with different merits depending on what your next goal is. With network stuff, it's perhaps easier to be confident. And I dare say one good doc is better than 10,000 conflicting docs.

    Speaking of conflicting advice...

    Actually, although in the manual that's supposed to mean 'no limit' (so it would make logical sense at the start of the file), in real world use you may find you need to reset this again at the start of each loop so it'll reset the no-limit again and won't time out. (This I know from other PHP processing - I'm not yet using DAC but the comments here are fascinating).
     
  12. Aaron Clifford United Kingdom

    Aaron Clifford Well-Known Member

    Joined:
    Jun 2010
    Posts:
    2,236
    Likes Received:
    29
    Where is my conflicting advice?
     
  13. Aaron Clifford United Kingdom

    Aaron Clifford Well-Known Member

    Joined:
    Jun 2010
    Posts:
    2,236
    Likes Received:
    29
    To add to the above, I've used the timeout code on my own scripts that run for 24 hours a day without issue. So I was making my suggestion based on my own experience with the DAC.
     
  14. PaulGregory

    PaulGregory Active Member

    Joined:
    Jun 2013
    Posts:
    230
    Likes Received:
    2
    Sorry Aarron, I made a sloppy link between 'more documentation = conflicting advice' and my refinement of your suggestion. There is a minor conflict between "top of file" and "inside loop" but I mainly meant that my refinement contradicts the official part of the PHP documentation.

    Not everyone's PHP is processed in the exact same way - I appreciate my contribution seems like voodoo PHP but it has worked for me when processing big arrays of product stock data (particularly on WAMP). If you are in an environment that sets a limit on an unlimited timelimit, resetting the timelimit regularly works. If you are in an environment that works as the docs suggest, it doesn't make a difference.

    I appreciate this is not based on DAC experience, but it is based on WAMP experience and given that the OP appears to still have a timeout problem and is using WAMP I thought it was worth a mention.
     
  15. Aaron Clifford United Kingdom

    Aaron Clifford Well-Known Member

    Joined:
    Jun 2010
    Posts:
    2,236
    Likes Received:
    29
    Ah right, I wasn't neigh saying what you said :).
     
Thread Status:
Not open for further replies.

Share This Page