Nominet Zone File Released in May

[Edwin]

Could somebody kindly post an example line or two from the files, obfuscating the actual domains? Say one for a .uk domain and one for a .co.uk. That way, people monitoring this thread have some idea of the level of detail available without having to download the whole thing...

 

[invincible]

Yes it's a full zone file.

There's a database dump simply containing all registered domain names.

Then there's seperate zone files for all the extensions continuing the active domain names at the time the snapshot was taken. Each domain name is listed along with its name server. Where a domain name has more than one name server (quite usual) that domain name is listed multiple times with each differing name server on a different line. There's nothing else of use.

 

[jmcc]

Could somebody kindly post an example line or two from the files, obfuscating the actual domains? Say one for a .uk domain and one for a .co.uk. That way, people monitoring this thread have some idea of the level of detail available without having to download the whole thing...

Would be very helpful, Edwin,
The full .uk zonefile (including subs) should be around 27 million or more lines based on a quick line count of the .org zone.

Regards...jmcc

 

[armistice]

OK Edwin,

Todays zip file is ukdata-20160516.zip its 232mb compressed.
When unzipped it contains:
1.2G May 16 08:32 co.uk-20160516
211M May 16 09:07 db-dump-20160516.csv
983K May 16 08:32 ltd.uk-20160516
11M May 16 08:32 me.uk-20160516
80K May 16 08:32 net.uk-20160516
64M May 16 08:32 org.uk-20160516
51K May 16 08:32 plc.uk-20160516
4.4M May 16 08:32 sch.uk-20160516
68M May 16 08:30 uk-20160516

The first few lines of each file are: the NS records for the namespace.
Following that one line per domain nameserver record. This means that a domain with 2 nameservers has two lines. heres the first few lines from me.uk
0.me.uk. 172800 IN NS art.ns.cloudflare.com.
0.me.uk. 172800 IN NS beth.ns.cloudflare.com.
00.me.uk. 172800 IN NS ns1.chesternet.com.
00.me.uk. 172800 IN NS ns2.chesternet.com.
000.me.uk. 172800 IN NS ns33.domaincontrol.com.
000.me.uk. 172800 IN NS ns34.domaincontrol.com.
0003.me.uk. 172800 IN NS ns-uk.1and1-dns.co.uk.
0003.me.uk. 172800 IN NS ns-uk.1and1-dns.biz.
0003.me.uk. 172800 IN NS ns-uk.1and1-dns.com.
0003.me.uk. 172800 IN NS ns-uk.1and1-dns.org.

 

[jmcc]

Looks like a relatively ordinary zone file. (There are some real nightmares like the .name zone). No major problems parsing it from what can be seen above.

Regards...jmcc

 

[armistice]

The db-dump file is a comma separated list of domains in the following line format:
<domain text> , extension

So for example acorndomains.co.uk appears as acorndomains,co.uk

One per line,
Interestingly its not sorted alphabetically, so I bet some interesting theories as to the significance of the ordering will follow shortly

 

[ian]

Doesn't hold much value to me personally, more interested in expiry dates and tags.

 

[invincible]

Doesn't hold much value to me personally, more interested in expiry dates and tags.

Build that from the DAC/delayed DAC.

 

[Edwin]

Thanks for posting the detail.

Seems to me that somebody with database skills could build a complete picture of the take-up of .uk over time by comparing .co.uk and .uk Whois records using DAC (it would take a few days worth of DAC quota to complete the comparison) which would show exactly how many .uk rights were taken up by .co.uk owners each day since .uk was launched.

In pseudocode, something like:
1) Does .uk have matching .co.uk?
2) If yes, then has .co.uk been continuously registered since before the cut-off date for .uk right of first refusal?
3) If yes, then .uk must have been registered by .co.uk registrant exercising that right

 

[redbird]

My last line within that is 10700061.

According to my import there are only 10700025 unique names, the discrepancy appears to be duplicates. How can that be?

e.g. domainfamily,co.uk on line 7013085 and 7010933. Albeit that the first one has uppercase "D" whereas second is lowercase, can someone please confirm I am not imagining this?

 

[invincible]

According to my import there are only 10700025 unique names, the discrepancy appears to be duplicates. How can that be?

e.g. domainfamily,co.uk on line 7013085 and 7010933. Albeit that the first one has uppercase "D" whereas second is lowercase, can someone please confirm I am not imagining this?

http://www.nominet.uk/whois/?query=domainfamily.co.uk#whois-results - probably a special case given who it's registered to and no expiration date.

 

[redbird]

http://www.nominet.uk/whois/?query=domainfamily.co.uk#whois-results - probably a special case given who it's registered to and no expiration date.

I did see who it was registered to (haven't checked the others) but even so I cannot think of any valid reason why there should be a duplicate.

 

[invincible]

I did see who it was registered to (haven't checked the others) but even so I cannot think of any valid reason why there should be a duplicate.

There is an exception and unsurprisingly it's registered to the registry without an expiration date. I probably don't need to understand why under those circumstances. You might want to check the others and the reasons for those might become self explanatory too. Is this only within the database dump rather than the zone?

 

[redbird]

There is an exception and unsurprisingly it's registered to the registry without an expiration date. I probably don't need to understand why under those circumstances. You might want to check the others and the reasons for those might become self explanatory too. Is this only within the database dump rather than the zone?

I haven't processed the zones yet, so yes just the dump, I am currently in the process of identifying the others.

Not a problem for me as I'm using domain+extension as a primary key (was just domain, hence why I was originally 1million short!) so I won't end up with duplicates, I was just surprised that my count wasn't an exact match.

I'm sure tomorrow's file will be correct

They are all same registrant and most seem to have a theme (connection to registrant is not obvious) and most also expire on Thursday

 

[PoshTiger]

What on earth are you guys going to do with this data anyway?

There's so much of it with the majority being worthless I don't think I'll bother.

 

[invincible]

What on earth are you guys going to do with this data anyway?

There's so much of it with the majority being worthless I don't think I'll bother.

Read through it by eye (dude).

 

[Skinner]

Most are going to run it through the dac to create absolute drop lists. If you can't code, its almost useless to you other than to try and find potential buyers.

There are a few other uses too, don't wanna help potential rivals

 

[BurtyB]

What on earth are you guys going to do with this data anyway?

The same as the other zone files for me, to keep my DNS history site updated. I'm sure Nominet will also be pleased with the drop in domain probing DNS queries .

 

[jmcc]

What on earth are you guys going to do with this data anyway?

There's so much of it with the majority being worthless I don't think I'll bother.

Add the unknowns to the 460 million or so active and deleted domain names tracked here amongst other things.

Regards...jmcc

 

[timter51]

Sorry to be a complete mug here... where can I download the dump??