Attempting to use different generation modules, or even same part numbers that were made with a significant time between manufacture is highly likely, in fact, is most probable, to not work. Chip density, voltage requirements and a variety of other design differences can all affect compatibility. Even if they do play together, they may not play in dual channel.
It's always best to used matched sets or when that's not possible, at the very least use identical part numbers from the same generation, not from different generations as sometimes the part numbers get recycled.
If gaming is your main priority you're unlikely to see any performance difference using more than 8GB of RAM anyhow. Only seriously demanding applications will benefit from 16 or 32GB of RAM unless you're running virtual machines, which is an entirely different ballgame.