Just registered to say that ghuiop23's solution of going down to 59HZ (https://forums.tomshardware.com/threads/rtx-3080-flickering-on-top-of-screen-using-dp.3650406/post-22153020) works :)
Thanks a lot! Now I can use the HDMI Port for my switch like it's supposed to work.