No, you didn't get the wrong information from your vet. Deafness is indeed linked to colour. More accurately, it is linked to
lack of colour. The white coat colour in boxers isn't a colour, per se, it is an extreme lack of pigmentation. Not albinoism, but lack of pigment.
That lack of pigment in the inner ear is what leads to deafness. Lack of pigment leads to suppression of blood supply to the cochlea, which results in deafness. Same as in dalmations, only more so
A 'white' boxer with patches of colour around the ears is less likely to be deaf than one that is fully white because the inner ear is more likely to have normal pigment. So your vet was quite correct.
Here is a very useful thread which discusses this in some detail. Read the information posted by Marimat - it is an excellent and easily understood account of the issues. Technically the thread is about why white boxers should never be bred (because that deafness is actually hereditary) but it will give you a very good overview of deafness in whites, and in the boxer breed as a whole.
http://www.boxerworld.com/forums/showthread.php?t=34760
If you'd like more scientific information, then this article by Dr Bruce Cattanach (boxer breeder and geneticist) also gives some excellent information. It is actually about Dalmations, but the information is factual irrespective of breed

The white colour in boxers, incidentally is also caused by the extreme white spotting allele (sw).
http://www.steynmere.com/DALM_DEAFNESS.html
Incidentally, the early bulldog was just one of many "contributors" to the creation of the boxer. You can read a very brief account of the development of the breed (concentrating on those that are the ancestors of boxer in the US) on the American Boxer Club's site
http://americanboxerclub.org/boxer_history.html