Fewer modules is always better. Regardless of what each modules size is, each module uses a specific voltage. The more modules there are, the more strain on the memory controller. Two modules uses less voltage than four modules, therefore less stress and better stability. You still want to maintain dual channel operation though, which is why it's recommended to buy in pairs. The only time you want to use more memory banks is when you need more memory but the slots that are populated already have the largest possible module capacity installed.
If you want 4GB, use 2 x2GB, for 8 use 2 x4GB, for 16 use 2 x8GB, etc.
If the board only supports 4GB modules in each slot, but you want 16GB, that's when you would use four modules. If each slot supported 8GB modules, you would want to use only two slots to make your 16GB.