Vertical mount is only for eyecandy, while it can produce all sorts of issues.
E.g
Main issue is using the riser cable. There have been countless of instances where riser cable alone, either doesn't make proper connection to the GPU (reduced GPU performance) or GPU isn't detected at all. But once riser cable is removed from the system, GPU works fine.
Another issue is airflow, since many riser brackets put the GPU close to the transparent side panel. But with this, you'd be block GPU fans and essentially choking the fans for airflow intake. Thus, GPU temps are far higher than normal.
Most vertical GPUs are located at the bottom left of the MoBo, but i've seen different GPU locations as well. E.g:
But with that setup, my question would be, how can you get your monitor cable to the GPU? Since GPU is dead smack in the middle of the PC and monitor ports are at the PCI bracket of the GPU.
GPU orientation doesn't matter.
Sure, in horizontal (normal) mounting the layers from top to bottom would be:
GPU backplate
GPU PCB
GPU die
GPU cooler cold plate
GPU heatsink
GPU fans
But if you look CPU coolers, namely air coolers, there, the common configuration would be, from right to left;
heatsink and fans - CPU cooler cold plate - CPU - CPU socket - MoBo PCB - MoBo backplate - MoBo tray (PC case).
And now, if you flip the GPU into vertical, it has the same order of layers, from right to left;
GPU fans - GPU heatsink - GPU cooler cold plate - GPU die - GPU PCB - GPU backplate.