Bounty,
You're way off. Putting it on the chipset is COMPLETELY different from putting it on the processor die. There is no advantage of putting it on the chipset, that's why I said it makes no sense that they continue to do it. The processor is different.
Dave K, you can add pins pretty easily, just like they always do. Look at LGA775. They went from 478 to 775 for the same processors. So, that's one alternative. How about all the extra lines on the motherboard for extra x16 processors? In some ways, you could simplify it by offering fewer lanes, since they aren't needed. The intercommunication between the processor and the GPU would, naturally, be a lot faster too. You'd have less traffic through the chipsets, which would help. You wouldn't need memory controllers all over the place either. You could also integrate the functions of the GPU with a processor as well, although I think in the near term this is less likely. But, surely it's also something designers have thought about too. There's a lot of potential with this.
It's not simple, but it's not very difficult either. Clearly, it's happening, and it's happening soon. Initially, because of die considerations, it will start at the low end, but will surely move up. As more die area becomes available, the GPUs will get bigger, and their inherent efficiency will give them an edge. Also, consider if you do have two memory banks, one for the GPU and one for the CPU (It's a bit un-Von Neumann, but ...), with creative use, you could use the extra bandwidth of the other while it's not busy. I personally think you'll see the same memory used, although there might be a cache, because it opens up so many opportunities for greater flexibility. On the other hand, you'd give up advantages of specific memory for graphics, but they use DDR3 a lot for cards, so it's not so bad, obviously.
Considering the inefficiency of discrete cards getting data, instructions, etc..., and also having to use main memory when the card doesn't have enough, I think it's inevitable.
You would lose flexibility, and I don't think the processor makers would offer so many CPU/GPU combinations that you'd eliminate that downfall, but it's a small negative considering how many people do it, and the advantages. Most people replace whole machines.