well....this can get really lengthy.... but here is a general case of a male for example..
Testosterone comes from the process of the production of GnRH in the hypothalmus which gets released into the pituitary gland which in turn causes LH (leutinizing hormone) release from the pituitary which cause the leydig cells in the testes to produce testosterone. Remember what testosterone is and does. Testosterone is the androgenic hormone responsible for growth and development of sex and reproductive organs, including the penis, testicles, scrotum, prostate, and seminal vesicles. It also relates to secondary male sex characteristics such as musculature, bone mass, fat distribution, etc. etc. Thus without enough testosterone in the system prior to neutering/castration of an animal you can see some changes HOWEVER this would have to be a VERY young animal. You are not likely to see big differences as most would be in possibly some hormone levels or something but at an age that any person or vet would neuter an animal there should have been adequite hormone production.... I am not saying either that leaving a male intact will make him grow bigger or anything because that hasn't been proven either, but to say there is no chance of a problem doing it too young isn't correct in my opinion either....