The neighbor's behavior concerns me, too. What other liberties might he take when you aren't around to see? Setting that aside, though, could he be misinterpreting a play bow? You said your dog is all wiggles and shows affection to the neighbor while you are there to supervise. Could your dog be play bowing? I know my dogs will bow low and almost pounce from side to side and even forward when initiating play this way. They are terribly vocal and will grumble and growl but it's all in play. My male will resort to a demanding bark when frustrated that could be interpreted as aggressive if you didn't know better.
I would ask the neighbor's help and watch from a window while he approaches your dog to see the behavior and judge for yourself. If it's just play, explain it to your neighbor. Boxers are exhuberant and do have a different play style. It could be that his dog in fact does some of the same behaviors, but since he has a Chi, he is not used to seeing it come from a bigger, louder breed.
It could be fear. Some fearful dogs are submissive and some become fear-aggressive. My rescued female is fear-aggressive and will bark and lunge at strange people and things. I've had people exclaim "My! She's so protective." and I have to laugh at their misinterpretation of her behavior. If she were truely protective, she wouldn't be putting me between herself and the scary thing! She has mastered the art of lunging forward while moving backward away from scary things. If your dog is showing fear by either submitting or aggressing, a good program of gradually socializing him to strangers will help change the behavior. In the meantime, explain to your neighbor that you are working on the behavior and you don't want your dog to practice inappropriate behavior when your not there to interrupt it, so could he please not approach the fence when you are not there to supervise.
If the behavior seems harmless to you, or your neighbor is going crazy and seems to be making the whole thing up but insists on continuing to approach your dog anyway, you can always tell him that you've heard of a rash of dog thefts in your area where dogs are stolen right from their yards. Tell him you don't want your dog to learn to approach people at the fence or accept treats and petting from strangers without you there and so could he please no longer approach your dog when he is alone in the yard.