Yes.
It doesn't matter in which order you unplug and replug the cables.
But for me, i've found it easier to connect all required cables to the PSU 1st and then route each and every one to the proper place where they need to go.
It ONLY works when PSU is plugged to the mains. But since one of your tasks is replacing PSU, where you have to remove the main power cable between PSU and wall outlet, you can't ground yourself by just touching any metal part of the PSU (or PC case in that matter), since there is no ground connection.
All one needs to know about ESD and PCs + plethora of info about ESD, is all seen/heard here:
View: https://www.youtube.com/watch?v=m54ifTCgtN4
I suggest you watch it. Steve has made very informative video about ESD.
I bought these gloves as well.
https://www.amazon.co.jp/-/en/dp/B00AXTVONU?psc=1&smid=AN1VRQENFRJN5&ref_=chk_typ_inl_img
I hope that this protection would be good enough. It seems that wireless wristbands are scam so I bought the ones with the cable. I'm not exactly sure where should I attach the cable, it seems that it should be connected to the metal part of the pc case according to the Internet.
Just in case, I won't wear socks and will not stand on the carpet, I hope that this is enough....
Btw, my new graphics card just arrived and there is nothing but the main unit inside. I presume that the cables that I need to use are the ones that are included with the new PSU along with the same cable that connects to the monitor.