There are occasional differences when it comes to patching or modding a handful of games, but as digital distribution has eclipsed the sale of physical media, you'd be hard pressed to come across any difference with more recent titles. In terms of shear performance, you'll not notice a difference. You will have Steam software running in the background, but modern computers are all carrying dozens of background services and applications at all times; it's not going to matter.
The major drawbacks to Steam are that customers have almost no legal standing, customer support is extremely poor, and the moderation of the community is generally abusive—the Help and Support section of it, anyways. You can find yourself in all sorts of trouble if you publicly discuss a problem or highlight a problem with the service. Once every couple of years, Gabe Newell answers a customer's email and the marketing department has the press run the story to soften the company's image.
Otherwise, Steam is great as long as you don't have a problem.