I'm having the same issue. I posted an article on the raspberry pi reddit page but didn't get any response there (I'll paste the post below). In short, I am getting a peak of 200-300Mb/s download before my CPU usage goes to 100% and my speed falls to about 50-100Mb/s. Uploads are not affected and I get at least 600 each test, most times in the 800's without the same CPU usage. Ethernet and wifi connection occasionally dropping for 1-3 seconds, intermittently. In my post below, I outlined the troubleshooting I did without success
Post from Reddit:
"I own a 3B+, run pivpn and get speeds of approximately 70/70mb/s when connected to the VPN (ISP speeds are gig symmetrical; connected via ethernet; VPN cipher is aes-256-cbc; CPU usage at 80% when speeds hit 70's and limits speeds from going higher).
I bought a pi 4B, anticipating I'd get a sizable network performance boost given the better CPU, 4GB ram, and gig Ethernet. Running browser speed tests through chromium, I'm getting a peak of 200mb/s down with speeds ranging from 20-80mb/s at the end of the test (with 100% CPU usage at peak) and 600-800mb/s up on average, peaking at 900mb/s (with about 50% CPU usage at peak). On the download side, there is a clear link between CPU usage and network speeds and I don't know what's triggering this. I've factory reset my router, used many different speed tests and servers, all with the similar results (I can list if need be). I have run vcgencmd measure_temp with nothing higher than 54C during tests. I'm running the latest version of Buster. Other ethernet connected computer (desktop) able to hit 950/950mb/s. I was wondering if anyone else had noticed this as well and had any troubleshooting advice. I don't understand how the upload can go much faster, using less CPU while the download is limited so harshly.
Additionally pivpn speeds on the 4B are 100/100mb/s. I was hoping for more, having read the OpenVPN Performance Thread and wondered if my above issue was contributing. I know the pi doesn't have hardware accelerated encryption, but with all of the improvements between the 3B+ and the 4B, I would've expected a larger gain in throughout speeds. In context, CPU usage is less than 25% during the VPN speed tests where I achieve the 100/100 speeds."