Actually, people don't seem to understand how microphones work. Which is why you hear people who use one tapping on the keyboard with an open mic. This is all about sensitivity and capping.
If your sensitivity is low enough, the mic won't pick up your brother screaming in the other room. But will easily pick up what you are talking about, as long as you are using a directional mic/mouth piece.
Also, most sound doesn't go through walls, it goes through your door.