There are several reasons I can think of.
1. It's new. Older people (such as parents) didn't grow up with them, and so they don't fully understand them.
2. Video games have been blamed for violent behavior. Events like Columbine reinforce this, and there isn't a lot of evidence saying it's totally wrong, but most evidence just states that any behavior changes are minor and tend to subside after a few hours, anyway.
3. Video games are often anti-social, and parents like to see kids with lots of friends. Hence, go outside and play, make friends = happy parents, anything else= less happy parents.
4. Because some "evil" games, like the Grant Theft Auto series have been so highly publicized and criticized, less-informed parents tend to believe all games are as violent, lewd, and contain similar "bad" elements.
Those are mostly opinions.