Unless it was transferred to you with a website and or mail or some other service on the domain that you expect to keep you should remove all DNS records. Even better, remove all name servers or set your own ones.
By not removing DNS records you are delegating control of your domain to others...