Smarter with designing scalable hardware? I don't think so! Intel had a larger customer base = more money for research and development and therefore they managed to compete with more price efficient products. This was coming from economies of scale and not because they were better. In fact, the Intel hardware with their 32bit CISC architecture was inferior to e.g. MIPS and SPARC that used 64bit RISC architectures with better pipelining and much faster bus/memory speeds. But Intel hardware was also a lot cheaper.
The Intel stuff was shit, but with the help of a lot of R&D they managed to overcome a lot of those hurdles. I'm not sure exactly when but they did a total redesign "under the hood" and now those x86 instructions are merely a layer on top of an internal instruction set of what they call "microcode". But this instruction set has been updated with quite a lot of extensions that render many of the old instructions redundant and obsolete. I think we are at AVX2.0 now. So yes, I'm aware of their redesign...
The current instruction set is a performance hurdle for Intel still to this day, how big a hurdle it is is hard to tell. I think one shouldn't rule out the possibility that one day the x86/x64 might get replaced by more efficient architectures in the future. The ARM CPU can gain market share, I think it is still inferior to Intel in server environments but it is developing as well and future ARM64 technologies may come with what is required for server environments. There is a lot of talk about power efficient ARM-based servers right now. The thing is that Intel has so far not yet managed to penetrate the mobile market and that could one day spell the end of Intel's architectures.