+1 the above posts.
The concern is that your dog will pick up an illness (like parvo) that he has not yet been vaccinated for. This is a valid concern in places like a dog park, where there are lots of dogs around, and lots of stuff like dog feces laying around that might transmit such diseases. Your own backyard, however, is unlikely to pose a hazard. Just keep an eye on what your dog might get into out there, and you probably don't want your dog hanging around with "unknown" dogs. If you've got some redneck neighbors who let their dogs just run around, that's the kind of thing you want to watch out for.
I haven't done a ton of research into this, so I won't offer advice one way or the other, but you might want to read up a bit and decide how you want to weigh concerns for keeping your new puppy isolated to prevent disease vs. concerns for providing your puppy opportunities for healthy socialization with other dogs. Not having your puppy around other dogs is a good thing for susceptibility to illness since he hasn't had all of his shots, but on the other hand, by waiting around until he is old enough to have had all of his shots, you might miss out on some windows of time in which it is good for the pup to learn socialization skills from older dogs. If you have other dogs, this probably isn't as much of a concern, but if you do not and you know someone with healthy, well balanced dogs, you might want to set up some play dates. And you certainly will want to do this once the puppy is fully vaccinated. I think there are some articles on this site about the stages of puppy development, which will help you make decisions about how to raise a healthy puppy that will be a friend to everyone he meets and bring lots of joy to your life.
Congrats on the new pup!!!! They're a total pain in the butt, but also a complete joy : ).