Hi Michael,
Thanks for the background info. I have not been in the copyright situation wrt images, but plenty of times with other things. If you are mercenary, and if you think you can make more from the private sales, elsewhere, and he has added the usage restrictions after giving you permission to use his location, then it could depend on who has the best lawyer. But, as it is not like that. I think he may probably only want to prevent his rather special prints being readily duplicated. Why not find out what he is really wanting, and then possibly agree to what he has asked? Wealthy friends are better than wealthy enemies (guess he's wealthy, as he's buying your prints ;-)
Then, go to the distant hills, at dawn, and take another pano, so he can see his building. Sell him that too, and he has no hold over that image, and he has paid you to practice on the ones you have so far.
Best wishes,
Ray