Bulk Check Last Archive.org Entry

[baldidiot]

Is there any tool or method to look up the last time there was an entry on the wayback machine (ie: archive.org) for a set of domains?

I know that the UK Backorder droplist shows the first time it was archived, but I'm interested in the most recent time. Ie: How recently was a site seen on archive.org for a set of domains.

I basically just want to be able to sort through a pile of domains to see which had a site on them recently and which didn't. I'm aware that an archive.org entry doesn't necessarily mean there was a site, but it's better than nothing and just a tool to slim down some lists a bit.

 

[ukbackorder]

Just use the url below with the domain and today's datestamp eg 20240314 (14th March 2024)

https://archive.org/wayback/available?url=DOMAIN&timestamp=DATESTAMP

Example for acorndomains:
https://archive.org/wayback/available?url=acorndomains.co.uk&timestamp=20240314

The json response contains the last closest visit so in this case:
{"url": "acorndomains.co.uk", "archived_snapshots": {"closest": {"status": "200", "available": true, "url": "http://web.archive.org/web/20240210091456/https://www.acorndomains.co.uk/", "timestamp": "20240210091456"}}, "timestamp": "20240314"}

The answer is that timestamp in the response '20240210091456' Which is 09:14:56 10/02/2024 as the format is YYYYMMDDHHiiss
Note that is the last time a URL was visited and responded with a page

 

[RB5]

If you don't want to do the above approach, you can also use a web scraping tool (Screaming Frog is my go-to) and grab the details from the details page such as: https://web.archive.org/details/http://acorndomains.co.uk/

 

[baldidiot]

Some very useful advice, thanks both - I think that should give me what I need!