Nominet Zone File Released in May

[armistice]

Any one know what software should be used to open these files???

They are text files inside the zip, you can view them with any text viewer (although since the files are big notepad won't work well ).
You could import them into sql server, or even excel. Depends what you want to do with the data.

 

[ian]

They are text files inside the zip, you can view them with any text viewer (although since the files are big notepad won't work well ).
You could import them into sql server, or even excel. Depends what you want to do with the data.

Tried all that, too big to open in any mainstream software and excel struggles to import more than just the letter A's!

 

[redbird]

According to Nominet stats http://www.nominet.uk/news/reports-statistics/uk-register-statistics-2016/ there should be around 10.5m domains, I am only seeing less than 9.8m. Anyone else see more?

 

[redbird]

9,783,544 in the csv dump file. I thought that was all of them?
Maybe my import into msql failed somewhere. I used LOAD DATA LOCAL INFILE

 

[AssetDomains]

Tried all that, too big to open in any mainstream software and excel struggles to import more than just the letter A's!

Haven't tried it on the Zone but bigdump was handy when I was looking to import the uk postcode data that runs into millions of records
http://www.ozerov.de/bigdump/

 

[redbird]

Haven't tried it on the Zone but bigdump was handy when I was looking to import the uk postcode data that runs into millions of records
http://www.ozerov.de/bigdump/

Is that level of postcode data publicly available, appreciate a link if free?

 

[AssetDomains]

Is that level of postcode data publicly available, appreciate a link if free?

You cant get the official royalmail one free but there are open source alternatives if you google

https://www.freemaptools.com/download-uk-postcode-lat-lng.htm I think I used

 

[ian]

Even the provided CSV file complains about size and only shows around 1mil results.

 

[ian]

Yes. Sorry. Too advanced for you.

Thanks David, great help as usual; you're a credit to Acorn

For anyone else looking for a simple solution, this text editor can cope with the file size:
https://www.emeditor.com/download/

 

[ian]

I guess you're attempting to open in Excel? If you're on Windows, grab something like Notepad++ to view the whole file.

Notepad++ suffers the same issue it seems. Will look at the config for it when I have more time. Thanks.

 

[ian]

Notepad++ works fine for me.

.uk file size is much smaller than the .co.uk

 

[jmcc]

You might do better using Linux with vi. TextWrangler for Mac imports them. Clearly some regular expressions will need to be performed on the data to remove the unnecessary parts before processing.

If it follows the usual zonefile format, then it should be possible to use Awk or Perl to pull out the domain names if that's all that is required. However it will be necessary to filter out any non-domain name data.

Regards...jmcc

 

[jmcc]

So is the zone file the full zone file complete with nameservers and glue records?

Regards...jmcc

 

[Edwin]

Could somebody kindly post an example line or two from the files, obfuscating the actual domains? Say one for a .uk domain and one for a .co.uk. That way, people monitoring this thread have some idea of the level of detail available without having to download the whole thing...

 

[jmcc]

Could somebody kindly post an example line or two from the files, obfuscating the actual domains? Say one for a .uk domain and one for a .co.uk. That way, people monitoring this thread have some idea of the level of detail available without having to download the whole thing...

Would be very helpful, Edwin,
The full .uk zonefile (including subs) should be around 27 million or more lines based on a quick line count of the .org zone.

Regards...jmcc

 

[armistice]

OK Edwin,

Todays zip file is ukdata-20160516.zip its 232mb compressed.
When unzipped it contains:
1.2G May 16 08:32 co.uk-20160516
211M May 16 09:07 db-dump-20160516.csv
983K May 16 08:32 ltd.uk-20160516
11M May 16 08:32 me.uk-20160516
80K May 16 08:32 net.uk-20160516
64M May 16 08:32 org.uk-20160516
51K May 16 08:32 plc.uk-20160516
4.4M May 16 08:32 sch.uk-20160516
68M May 16 08:30 uk-20160516

The first few lines of each file are: the NS records for the namespace.
Following that one line per domain nameserver record. This means that a domain with 2 nameservers has two lines. heres the first few lines from me.uk
0.me.uk. 172800 IN NS art.ns.cloudflare.com.
0.me.uk. 172800 IN NS beth.ns.cloudflare.com.
00.me.uk. 172800 IN NS ns1.chesternet.com.
00.me.uk. 172800 IN NS ns2.chesternet.com.
000.me.uk. 172800 IN NS ns33.domaincontrol.com.
000.me.uk. 172800 IN NS ns34.domaincontrol.com.
0003.me.uk. 172800 IN NS ns-uk.1and1-dns.co.uk.
0003.me.uk. 172800 IN NS ns-uk.1and1-dns.biz.
0003.me.uk. 172800 IN NS ns-uk.1and1-dns.com.
0003.me.uk. 172800 IN NS ns-uk.1and1-dns.org.

 

[jmcc]

Looks like a relatively ordinary zone file. (There are some real nightmares like the .name zone). No major problems parsing it from what can be seen above.

Regards...jmcc

 

[armistice]

The db-dump file is a comma separated list of domains in the following line format:
<domain text> , extension

So for example acorndomains.co.uk appears as acorndomains,co.uk

One per line,
Interestingly its not sorted alphabetically, so I bet some interesting theories as to the significance of the ordering will follow shortly

 

[ian]

Doesn't hold much value to me personally, more interested in expiry dates and tags.

 

[Edwin]

Thanks for posting the detail.

Seems to me that somebody with database skills could build a complete picture of the take-up of .uk over time by comparing .co.uk and .uk Whois records using DAC (it would take a few days worth of DAC quota to complete the comparison) which would show exactly how many .uk rights were taken up by .co.uk owners each day since .uk was launched.

In pseudocode, something like:
1) Does .uk have matching .co.uk?
2) If yes, then has .co.uk been continuously registered since before the cut-off date for .uk right of first refusal?
3) If yes, then .uk must have been registered by .co.uk registrant exercising that right