Windows XP is still so popular because Windows 7 has been on the market for only around six months, and Vista was launched with numerous bugs that took some time for them to sort out. Windows Vista never really recovered from the poor reputation it gained at launch from the bugs present in the OS not to mention all the buggy drivers that were out at the time (Nvidia and Creative spring to mind).
As for Windows Vista/Seven and better hardware support:
Memory management:
http://www.microsoft.com/whdc/system/sysinternals/memmgt.mspx
Multicore CPUs:
http://developer.amd.com/documentation/articles/pages/282007123.aspx
SSDs (Win7, rather than Vista)
http://download.microsoft.com/download/5/E/6/5E66B27B-988B-4F50-AF3A-C2FF1E62180F/COR-T558_WH08.pptx
Better multicore support resulting in better system performance with multicore and mutlithreaded CPUs. We've got six core CPUs here now for desktop machines, and with HyperThreading that's support for up to twelve threads. We've got machines with more than 4GB ram, Windows will use Superfetch to further improve system responsiveness and performance. We've got SSDs that dramatically improve system performance and Windows 7 has been optimised for SSDs, right out of the box.
Just as another example of how far Windows has come along in recent years, I was reinstalling my system a few months ago With a dual boot XP/Win7 setup (3Dmark 2001 and Aquamark love Windows XP) on my Core i7 setup. My install drive is a pair of WD Raptors in striped RAID configuration. Windows Seven was dead easy to install, I just stuck the Intel RAID drivers on a USB flash drive and the Windows setup detected and used them. Not so easy with XP. I can't just put them on a flash drive. Fine then, to be retro I pulled out my USB floppy drive and put the RAID drivers on a floppy disk. Sorry, can't do that either since my USB floppy drive isn't one of the two models supported by XP. Well fine then. I'll take the side panel off of my case, get an oldschool floppy drive, plug in the appropriate cables, enable the floppy drive in the BIOS then do it that way.
All that trouble just to read a ******* RAID driver. To throw fuel on the fire if I had just used Microsoft's default RAID driver in Windows Seven I wouldn't have had to do anything at all to make it work.
Even if you don't care for the Aero UI (I've yet to meet anyone who does't prefer it to the blue and green fisher fisher paykel inspired UI that XP comes with by default) the crazy thing is that you can turn it off. Who would have thought of that? Of course Aero allows you to take advantage of GPU acceleration in applications. Internet Explorer 9, and upcoming versions of Firefox will support GPU acceleration in the browser. That's going to speed things up markedly and enable some cool new things, like in this link below.
http://channel9.msdn.com/posts/Charles/IE-9-Surfing-on-the-GPU-with-D2D/
I actually agree with you with regards to the gadgets, I don't find them that useful at all. But it's no problem sine they are turned off by default.
The snap features enables me to easily work with multiple applications open on my screen at once. Rather than manually resizing my applications manually and then dragging them to where I want them I can quickly drag the window to the side of the screen and let the mouse button go and they snap to the side. It's incredibly useful on my 30" monitor since even with two apps side by side each still has 1280 pixels wide of space. It's a very simple features, there's not much to it, but I find myself using it all the time. The same can be said for the ability to drag a window to the top of the screen to maximize it, simple but very useful IMO.
The start menu, having a small search bar down there is incredibly useful too. In XP if I wanted to hunt for an application I only occasionally used (rather than something I use regularly, which would have a shortcut handy) I'd go into the programs section and search it down. In Windows 7 I just type the first few characters of that application's title and the system finds it for me instantly. The same if I need to quickly find a document. It's a simple addition that makes a big difference.
The new taskbar in Win7 also allows me to work with multiple apps a lot easier as well. If I'm listening to music in WMP for instance, and I want to access the controls I just hover the cursor over the WMP icon and I can access the controls right from there. If I want to access more sophisticated controls for WMP, or for another program like Word, or IE or any other application that has been coded to support the feature then I can just right click the icon. I can perform many simple tasks in programs this way, without actually having to bring the program up. It's just done from the taskbar. I can rearrange the icons on the taskbar if I want to. Positioning the cursor over an icon gives me a thumbnail display of that application, allowing me to more easily find what I'm after. If I have multiple windows of a single application open then I just put my cursor over the icon and it'll show me a thumbnail for all the windows for that application allowing me to easily select the one I want.
These are real improvements that result in real productivity gains for me. Though since Windows 7 is bloatware I've been considering going back to Windows 95 since it has very low memory usage!