Increased response times to DAC today?

[timter51]

Has anyone else noticed a bump in response times to the DAC today? I've had a consistent 2ms+ increase on my normal response times since my script started first thing this morn. Need to know whether to go moaning to my host, or if it's at Nominet's end!

 

[Retired_member41]

Where's your host [emoji102]

 

[timter51]

Currently with Astutium. So no one else at all is seeing increased latency? A traceroute from my server to the DAC is as good as ever, but response times in my script have taken a nasty hit over the last two days I haven't changed a thing. So frustrating.

 

[Retired_member41]

Was it just increased times over trying to register the popular domains? If so it's quite common when they get swamped to get a late epp response.

 

[timter51]

Nope, it's a constant increase now, I'm seeing it today as well. Every single response is around 2ms+ more than before. This started 00:00 yesterday morning, everything had been running nicely for months :/

 

[Adam H]

Sorry to be off topic slightly but you say 2ms+ increase, can i ask what is considered to be "good" in terms of response times ?

Asking from non-drop catcher that is tempted become a member

 

[nick-harper]

Where are you trying to Trace Route / Ping? I can give it a go from here to see what it returns.

 

[Retired_member41]

He's saying his trace hasn't changed but his server and script is handling the response slower.

 

[Skinner]

Could it still be lost time bouncing around on the nom side of the firewall ?

 

[armistice]

I'm seeing slower response times to dac requests than usual, I think its routing / speed in nominet's network. I notice variations in response times which I assume are due to nom changing network routing, using backup servers etc.

From a astuitum test host (VPS) today :

Tracing the path to dac.nic.uk (213.248.242.37) on TCP port 3043, 30 hops max
1 XXXXXXX.vps.astuitum.com (XXXXXX) 0.029 ms 0.020 ms
2 transit2.as29527.net (95.128.48.42) 0.547 ms 0.491 ms 0.270 ms
3 peering2.as29527.net (95.128.48.44) 0.653 ms 0.622 ms 0.616 ms
4 dc1nombgprp01-linx.nominet.org.uk (195.66.224.61) 19.947 ms 6.660 ms 1.559 ms
5 nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2) 2.087 ms 2.010 ms 2.072 ms
6 nomfwlp01-ge-0-0-3.nominet.org.uk (213.248.252.250) 3.716 ms 2.879 ms 2.899 ms
7 lb-dac.nominet.org.uk (213.248.242.37) 4.610 ms 4.048 ms 4.037 ms
8 lb-dac.nominet.org.uk (213.248.242.37) [open] 5.114 ms 4.453 ms 4.466 ms

PING dc1nombgprp01-linx.nominet.org.uk (195.66.224.61) 56(84) bytes of data.
64 bytes from dc1nombgprp01-linx.nominet.org.uk (195.66.224.61): icmp_seq=1 ttl=61 time=2.73 ms
64 bytes from dc1nombgprp01-linx.nominet.org.uk (195.66.224.61): icmp_seq=2 ttl=61 time=1.70 ms
64 bytes from dc1nombgprp01-linx.nominet.org.uk (195.66.224.61): icmp_seq=3 ttl=61 time=2.76 ms
64 bytes from dc1nombgprp01-linx.nominet.org.uk (195.66.224.61): icmp_seq=4 ttl=61 time=1.77 ms
64 bytes from dc1nombgprp01-linx.nominet.org.uk (195.66.224.61): icmp_seq=5 ttl=61 time=1.68 ms
64 bytes from dc1nombgprp01-linx.nominet.org.uk (195.66.224.61): icmp_seq=6 ttl=61 time=1.72 ms
64 bytes from dc1nombgprp01-linx.nominet.org.uk (195.66.224.61): icmp_seq=7 ttl=61 time=1.74 ms


HPING dac.nic.uk (XXXXXXXX): S set, 40 headers + 0 data bytes
len=44 ip=213.248.242.37 ttl=57 DF id=0 sport=3043 flags=SA seq=0 win=14600 rtt=6.2 ms
len=44 ip=213.248.242.37 ttl=57 DF id=0 sport=3043 flags=SA seq=1 win=14600 rtt=6.4 ms
len=44 ip=213.248.242.37 ttl=57 DF id=0 sport=3043 flags=SA seq=2 win=14600 rtt=6.2 ms
len=44 ip=213.248.242.37 ttl=57 DF id=0 sport=3043 flags=SA seq=3 win=14600 rtt=6.2 ms
len=44 ip=213.248.242.37 ttl=57 DF id=0 sport=3043 flags=SA seq=4 win=14600 rtt=6.3 ms

I have a test script that times a dac request time, as follows:
Start Timer, Send DAC lookup request, receive DAC lookup response, stop timer.

Here are the results
Testing dac times
00.0193367
00.0066197
00.0069519
00.0066174
00.0067439
00.0069165
00.0068260
00.0066985
00.0069023
00.0067252
00.0096860
00.0067279
00.0065291
00.0068207
00.0068575
00.0067602
00.0066930
00.0065623
00.0072781
00.0069818
Average is 0.007561735

It would be interesting to see other hosts stats if anyone would care to share.

 

[timter51]

Thank you Armistice! What you've posted is nigh on exactly what I'm seeing, which is a relief to know it's not just me, thanks for taking the time to post that info. Nominet have told me they aren't aware of issues if their network but have asked for more information.

 

[timter51]

Response times have now dropped back to normal levels after I reconnected after my open connections were dropped around 7am this morning. There must have been some work going on behind the Nom firewall. How are you doing Armistice?

 

[ian]

There was an official update this morning at 7am

http://registrars.nominet.org.uk/news/system-announcements/13-may-2015-system-maintenance

Wonder what it has done to catching this time!

 

[armistice]

Response times have now dropped back to normal levels after I reconnected after my open connections were dropped around 7am this morning. There must have been some work going on behind the Nom firewall. How are you doing Armistice?

Just run now: 9.20 am

Testing dac times
00.0153086
00.0048974
00.0049932
00.0049142
00.0059609
00.0078986
00.0053645
00.0047565
00.0049146
00.0049234
00.0080193
00.0048083
00.0049102
00.0058979
00.0046937
00.0047759
00.0048630
00.0048701
00.0047765
00.0052902
Average is 0.00584185

 

[armistice]

ping to dac down 2ms this morning (compared to yesterday)

HPING dac.nic.uk (xxxxxxxx): S set, 40 headers + 0 data bytes
len=44 ip=213.248.242.37 ttl=57 DF id=0 sport=3043 flags=SA seq=0 win=14600 rtt=3.8 ms
len=44 ip=213.248.242.37 ttl=57 DF id=0 sport=3043 flags=SA seq=1 win=14600 rtt=4.1 ms
len=44 ip=213.248.242.37 ttl=57 DF id=0 sport=3043 flags=SA seq=2 win=14600 rtt=4.0 ms
len=44 ip=213.248.242.37 ttl=57 DF id=0 sport=3043 flags=SA seq=3 win=14600 rtt=4.2 ms
len=44 ip=213.248.242.37 ttl=57 DF id=0 sport=3043 flags=SA seq=4 win=14600 rtt=3.9 ms

ping to nominet border remains the same as yesterday

64 bytes from dc1nombgprp01-linx.nominet.org.uk (195.66.224.61): icmp_seq=10 ttl=61 time=1.69 ms
64 bytes from dc1nombgprp01-linx.nominet.org.uk (195.66.224.61): icmp_seq=11 ttl=61 time=1.73 ms
64 bytes from dc1nombgprp01-linx.nominet.org.uk (195.66.224.61): icmp_seq=12 ttl=61 time=1.78 ms
64 bytes from dc1nombgprp01-linx.nominet.org.uk (195.66.224.61): icmp_seq=13 ttl=61 time=1.73 ms
64 bytes from dc1nombgprp01-linx.nominet.org.uk (195.66.224.61): icmp_seq=14 ttl=61 time=1.79 ms

 

[timter51]

Excellent, looking good

 

[DarrenTSO]

The nature of BGP networks means that there can be routing changes and so on that can cause the path your packets take from your server to Nominet can vary. It could be as a result of maintenance - either at your provider, Nominet, or an upstream of either or somewhere in between. From a basic traceroute at the moment to dac.nic.uk though:-

# traceroute dac.nic.uk
traceroute to dac.nic.uk (213.248.242.37), 30 hops max, 60 byte packets
1 188.65.117.65 (188.65.117.65) 1.778 ms 1.825 ms 1.820 ms
2 uk.slo.ld5.201.r1.misp.co.uk (185.52.26.252) 0.254 ms 0.279 ms 0.278 ms
3 dc1nombgprp01-linx.nominet.org.uk (195.66.224.61) 0.273 ms 0.268 ms 0.264 ms
4 * * *
5 * * *
6 * * *
7 * * *
8 * * *

 

[armistice]

Very nice, any chance you could ping nomfwlp01-reth0-0.nominet.org.uk
for me?

Do you provide VPS on that box in ld5?

The nature of BGP networks means that there can be routing changes and so on that can cause the path your packets take from your server to Nominet can vary. It could be as a result of maintenance - either at your provider, Nominet, or an upstream of either or somewhere in between. From a basic traceroute at the moment to dac.nic.uk though:-

# traceroute dac.nic.uk
traceroute to dac.nic.uk (213.248.242.37), 30 hops max, 60 byte packets
1 188.65.117.65 (188.65.117.65) 1.778 ms 1.825 ms 1.820 ms
2 uk.slo.ld5.201.r1.misp.co.uk (185.52.26.252) 0.254 ms 0.279 ms 0.278 ms
3 dc1nombgprp01-linx.nominet.org.uk (195.66.224.61) 0.273 ms 0.268 ms 0.264 ms
4 * * *
5 * * *
6 * * *
7 * * *
8 * * *

 

[DarrenTSO]

Interestingly enough the latency climbs once you get past Nominet's gateway:-

# traceroute nomfwlp01-reth0-0.nominet.org.uk
traceroute to nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2), 30 hops max, 60 byte packets
1 91.198.165.65 (91.198.165.65) 1.710 ms 1.686 ms 1.678 ms
2 uk.slo.prgn.aggr.sw1.misp.co.uk (185.52.26.254) 1.276 ms 1.259 ms 1.150 ms
3 uk.slo.prgn.a9.r1.misp.co.uk (185.52.26.182) 0.263 ms 0.257 ms 0.245 ms
4 uk.slo.prgn.b9.r1.misp.co.uk (185.52.26.187) 0.323 ms 0.307 ms 0.278 ms
5 uk.slo.ld5.101.r1.misp.co.uk (185.52.26.135) 0.372 ms 0.359 ms 0.341 ms
6 dc1nombgprp01-linx.nominet.org.uk (195.66.224.61) 0.279 ms 0.306 ms 0.286 ms
7 nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2) 5.349 ms 5.139 ms 5.142 ms

I would guess that hop 6 is on the LINX in Equinix and then hop 7 might be over in Oxford?

That was actually from one of our boxes in our own datacentre but we have dark fibre back to LD5 where we pick up the LINX and so on.

 

[armistice]

Interesting, from my ast test host I'm getting

traceroute to nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2), 30 hops max, 60 byte packets
1 sovz1.vps.astuitum.com (91.227.124.10) 0.023 ms 0.006 ms 0.006 ms
2 transit2.as29527.net (95.128.48.42) 0.595 ms 0.606 ms 0.600 ms
3 peering2.as29527.net (95.128.48.44) 0.593 ms 0.584 ms 0.576 ms
4 dc1nombgprp01-linx.nominet.org.uk (195.66.224.61) 23.704 ms 23.667 ms 23.689 ms
5 nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2) 4.937 ms 4.925 ms 4.903 ms


But ping time are:
# ping nomfwlp01-reth0-0.nominet.org.uk
PING nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2) 56(84) bytes of data.
64 bytes from nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2): icmp_seq=1 ttl=60 time=2.20 ms
64 bytes from nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2): icmp_seq=2 ttl=60 time=2.44 ms
64 bytes from nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2): icmp_seq=3 ttl=60 time=2.36 ms
64 bytes from nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2): icmp_seq=4 ttl=60 time=2.29 ms
64 bytes from nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2): icmp_seq=5 ttl=60 time=2.29 ms
64 bytes from nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2): icmp_seq=6 ttl=60 time=2.27 ms

If you could run a ping from the server in your first post, I'd be interested to see the results.

Interestingly enough the latency climbs once you get past Nominet's gateway:-

# traceroute nomfwlp01-reth0-0.nominet.org.uk
traceroute to nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2), 30 hops max, 60 byte packets
1 91.198.165.65 (91.198.165.65) 1.710 ms 1.686 ms 1.678 ms
2 uk.slo.prgn.aggr.sw1.misp.co.uk (185.52.26.254) 1.276 ms 1.259 ms 1.150 ms
3 uk.slo.prgn.a9.r1.misp.co.uk (185.52.26.182) 0.263 ms 0.257 ms 0.245 ms
4 uk.slo.prgn.b9.r1.misp.co.uk (185.52.26.187) 0.323 ms 0.307 ms 0.278 ms
5 uk.slo.ld5.101.r1.misp.co.uk (185.52.26.135) 0.372 ms 0.359 ms 0.341 ms
6 dc1nombgprp01-linx.nominet.org.uk (195.66.224.61) 0.279 ms 0.306 ms 0.286 ms
7 nomfwlp01-reth0-0.nominet.org.uk (213.248.252.2) 5.349 ms 5.139 ms 5.142 ms

I would guess that hop 6 is on the LINX in Equinix and then hop 7 might be over in Oxford?

That was actually from one of our boxes in our own datacentre but we have dark fibre back to LD5 where we pick up the LINX and so on.