I wouldn't chalk it up to him being mad, dogs don't have the thought process to seek revenge.
If he does it everytime you leave him out of his crate, he could be defecating as a result of distress. I am not saying he has full blown Separation Anxiety, but he may be feeling the stress of going from a confined area that is familiar, like his crate, to being on his own with too much freedom. If you give him the whole run of the house, he honestly may not know what to do with himself and that is very stressful for a dog. What I would suggest is to start him out confined in one room, like the kitchen and work your way up to larger areas - at the same time, you will need to work on the timing as well. So if you leave him in the kitchen, start with being gone for 10 minutes, then 15, 20, 30, etc...if he relapses, then back up. If he does well, then let him have the living room and start over with the timing. You have to set him up for success or else he's not going to enjoy his freedom. Do a search on Separation Anxiety - you should come up with some other training tips on how to desensitize him to your leaving. Good luck!
