TLDR; if you just want it to work with Windows Speech Recognition (WSR), skip to the end and try the registry hack.
So... I just happened to be going through a clean install of Windows 10 for an entirely different issue and I retried my steps above with no success. I started going through and trying a myriad of different things while keeping very detailed notes.
At one point many uninstall/unplug/replug/reboot cycles in, I decided to try doing the setup via settings instead of launching my WSR shortcut for it. Got the same "Peter talks to his computer..." dialog, this time without any error pop-ups. Although it mentioned Cortana this time instead of WSR. Then I went back to control panel, skipped the mic setup, and went straight to train your computer. Was able to go through several screens without any problems (didn't complete it, was just testing). At that point, I started thinking that maybe WSR is just getting stuck on something...
Since my mic was obviously working with the default drivers, I decided to see if I could get WSR to quit being an idiot. After playing around in the registry, I found that simply changing one value did the trick. No more restarts/mucking around withh drivers.
So, assuming you just want it to work with WSR and it is already picking up sound in the dialogs I described above, the below command should do the trick. I'll probably end up doing another clean install sometime in the near future and will try to confirm if any mucking with the drivers is required at all or not prior to this. I suspect that the default driver might be just fine and WSR is the only one with the hang-up.
from command prompt run (all on one line):
PHP:
reg add "HKEY_CURRENT_USER\SOFTWARE\Microsoft\Speech\Preferences" /v "en-US_CompletedSpeechConfiguration" /t REG_DWORD /d "0x00000001" /f
(and then re-launch WSR)
EDIT: I'm also assuming that if you use a locale that is *NOT* en-US, then you may need to browse to the above in regedit and make the same change for whatever the corresponding value is named in your locale.