In theory, if the audio is coming from different sources, you can send it to different outputs. "Source" here is relative to the OS. It wouldn't work in your case since you're using Chrome for both audio streams - Google would have to modify Chrome to send different audio to different devices for different tabs. Though there might be a Chrome extension which lets you do it (regular Chrome and Chrome incognito mode are considered different browsers with independent extensions and settings).
https://chrome.google.com/webstore/detail/audiopick/gfhcppdamigjkficnjnhmnljljhagaha?hl=en
https://chrome.google.com/webstore/detail/youtube-output-devices/hecekbdnfioifkbcnemlpnhdfdmgoinj?hl=en-US
You might also be able to do it using a different browser for the second stream. In that case you'd need an extension like above to allow one of the browsers to send audio to a different device than the system default.
The one thing Windows cannot do with audio, to many people's consternation, is send the same audio stream to multiple outputs. It's only one input to one output, or multiple inputs (via mixing) to one output.