gareth_brown

Hopefully this is the right place to post this.

Why am I writing this post? I've been lurking around for a while and wanted to give something back, if I can.

Drop Catching is one of those things no one in the domain world wants to talk about. Well they won’t talk about their scripts! Why would they? It would be like Coke giving up their magic recipe.

So far it has been a 5 week project and I have lost count of how many hours it’s taken (and continue to take). It is still in development and currently off-line. Every day I tweak code, measure code speed and fine tune socket communication. It’s amazing how many hours you can spend measuring and analysing milliseconds. Okay, I am not going to give you any code but here are some things that I have learnt.

My software is currently written in C#, but I might move to C++ if it will provide faster timings. I am sure it won’t and hopefully below will explain why.
I will say up front, I do not know anyone here or anyone that Drop Catches, so I’ve been working in the dark. It’s been fun stumbling around trying to understand how it all works. To be honest it is surprising basic.

The process:

1. Repeatability query DAC for domain availability
2. If domain drops send EPP create request

My average timings:

These times can vary, not by a lot, but they do vary. The reason your timings will vary will be due to network loads at your data centre and those at Nominet. Nominet’s systems are going to be the slowest part of the system, so be careful what the time of day you test!

• Sending DAC query 0.0015ms
• Receiving DAC query 0.0020ms
• Sending EPP Request 0.05ms
• Receiving EPP Request 60ms
• From drop notification to send EPP Request 0.0002ms

Receiving the response back from the EPP is slow. I can only imagine their server is a) under load and b) their XML parsing and writing is probably not optimised. There is also a delay on the EPP communication due to the secure socket layer, so encryption is important here.

Servers – Good Network

The golden 2ms from Nominet. I have so far found it hard to be that slow, my DAC send and receive is 350 ticks. I have read this a lot, services boasting about being close to Nominet. Based on my tests, some network monitoring and the above timings Nominet’s servers and coding is your barrier. That is not to say you shouldn’t aim to buy or rent a server on a good network.

Servers – High Spec

You don’t need to high spec machine. Depending on how it is written, your code will probably only run 4 to 6 threads. I’m running a Windows 2008 Quad Core server with 2GB ram and an 80Gb hard drive. Any more would be a waste.

Servers – Only Run Essentials

Only run services that are required, turn everything else off. You don’t want your operating system starting threads the take priority over yours.

Optimising your code and keep track

Keep tweaking your code and never settle for working. I spent one hour rewriting the EPP socket and managed half communication time. Compress data. Remove variables and method calls that are not necessary. Keep yourself connected. Log your transactions, code executions but more importantly your exceptions. Recover from failure; no one likes a site down SMS at 3am! Remember to turn your logging off when live.

Finally are you feeling lucky?

If you base your calculations someone’s system EPP request send taking 5ms (which is slow but possible) you are only as good as your last loop. If your scan intervals are every 200ms and the domain hasn’t dropped you have 199ms before the next check. That’s a long time for other systems to have checked before that next check. Too long! Even at the maximum 60ms intervals, you’ll probably still miss the domain if it drops between scans. It’s all about luck, you have to hope your system pings just at the right time.

I don’t have time tonight but I could also talk about strategy - 60ms, 200ms, multi-checks, blasting, etc. Probably open that one for discussion.

brum

What a great post good luck gareth and all the best hope u get it all working soon

stevebrowne

good post I should dust off my catching stuff again.

__________________
Guarantor Mortgage | Instant Digital Cameras | SDXC Cards

dotNetFreak

Very interesting post - Thanks for sharing!

Do you mean:

Start Timer -> Send Query -> Read Response -> End Timer

= 0.00350ms?

If so, sorry, but I think your StopWatch class is broken

gareth_brown

Nothing would surprise me when working with MS products. No, you are right it would be impossible for it to send and receive that fast. The timings are based on the time the data is started to be received. As mentioned above, the check at Nominet can take a few milliseconds depending on the time of day.

I also found a really interesting trick that speeds up Nominet's DAC by about 20% but couldn't end up using it. I wasn't sure if it was a fluke, so I ran two connections and tested 200,000 times on each.

monaghan

The basics are easy - just simple network port IO, but as you say, the timings are the fun part and how you address the issue of when to poll and at what frequency. From experience, just when you think you have a pattern to work with, the next drop blows away that pattern.

Your server seems very high spec for the task, perhaps try porting to a Linux system as you can generally build a custom kernel with much more control over what you can switch off. The code is going to be in an idle "wait state" for most of the time. Don't forget a decent firewall as those nasty little hackers trying to break into your box will take valuable process time.

Where are you getting your "golden 2ms from Nominet" figure from? I'm seeing more than 2ms between some hops inside Nominet so I presume you are referring to time to the border router?

__________________
Alex Monaghan - Drop Catch & Drop Lists - PM for details
Online Accounting|Pine Furniture|Barbie

gareth_brown

The "golden 2ms from Nominet" are figures quoted by drop catch services. Because no one talks about their scripts its difficult to know what to expect with regards to timings. My point at the bottom is, it doesn't really matter. You either get there before someone else or you don't, with so many milliseconds of gap between each check 'how fast is mine' really doesn't matter. Of course if it takes a 1 second, build the create command, connect to the epp server and send the request you could be struggling!

Linux isn't my bag and for something that is currently a little personal project i'm not going to worry if i'm a millisecond or two slower. I am sure with 356 days in a year I should get lucky one day

monaghan

Luck is a big part of the process

My first catch system was in C# and sat much further from Nominet than I am now it used the Automaton (long before EPP was available) and it did all right for a 1st bash.

Keep tweaking and tweaking and you'll get there, I've had 2 LLL.co.uk's so far (not with the C# code though) so it is possible to beat "the big boys" if you keep at it. There are a lot more catchers today than when I built my first system, so you need to find something that the others haven't got.

__________________
Alex Monaghan - Drop Catch & Drop Lists - PM for details
Online Accounting|Pine Furniture|Barbie

newpbc

Thanks gareth great post gona look into geting my own tag sometime this year. how much would you say you've spent so far geting set up

__________________
Laterooms Wordpress Solution |

GlasgowBob

Great post Gareth, many thanks for sharing your experience getting set up.

I am considering getting my own tag as well this year. Would you say the administrative process with nominet is reasonably straight forward?