Question Slow connecting Remote Desktop with nonstandard port

Mar 12, 2019
I have a server and it was getting pounded by bots trying to login over remote access. I changed the port used from the standard 3389 to a very high port up in the 40,000s. I also blocked the standard port. Since doing so the login attempts have dropped to zero. A strange side effect is not it takes 20-30 seconds to establish the connection and prompt for the username/password. If i switch back to 3389 the initial connection speed is around 1 second. Once connected the throughput is what i would expect regardless of the port used. A ping is under 20ms. I can only assume that when I use the high port number that some internet routers don't pass it connection request through and when it times out the previous router tries again using a different destination router but I know of no way to verify the theory. This is only an annoyance and only while connecting but I'm curious as to what's happening. Is there any method to force a connection path for future connections once a valid path has been established?