I Agree with keys. But if you feel that bad and depending on if the domain is of value you could say I have invested money in this domain and am already paying for a site to be developed. He may then come back with an offer. And you haven't offered to sell it to him so evetything is cushty.
