A 12GB 5600 XT wouldn't really make any sense. That would drive its cost up higher than a 5700 (A couple models with dual-fan coolers can be found for as little as $310 currently), while providing no tangible benefits over that card. If you want more VRAM and additional performance, the 5700 already exists for that purpose, and can be had for not too much more than a 5600 XT.
If neither card is able to produce useful frame rates with RT enabled I'd say that's a draw.
While most of the games featuring raytracing so far have tried to show off the effects in ways that tend to substantially impact performance, it's possible to utilize such effects in a more limited manner to keep performance reasonable while still enhancing visuals. For example, that Crytek "Neon Noir" demo that's able to run on non-RTX hardware relatively well. It uses a number of shortcuts to achieve that, including things like reducing the resolution of raytraced reflections, only performing raytracing on certain objects in the scene and mixing RT with more traditional lighting effects based on viewing distance. These kinds of shortcuts can be used in combination with dedicated hardware as well, so enabling raytracing won't necessarily make all RTX games unplayable.
And of course, there's the question of what are "useful frame rates". In a competitive FPS, enabling raytraced lighting effects probably won't be desirable if they have a big impact on performance, as they do with current hardware. In something like a slower-paced adventure game though, frame rates dipping below 60fps isn't going to be as much of a problem, especially if one is using an adaptive-sync display. So, for many games, the bar for playable performance will be lower, and taking a performance hit for improved visuals might be preferred. In any case, the point is that the 2060 has the potential to provide better performance with certain settings enabled down the line, as it includes some additional hardware not found in AMD's 5000-series. Whether that pans out or not remains to be seen, but I would hardly say there's any solid evidence indicating that the 5600 XT would pull ahead over time.