Normally, I'd suggest never pairing a 5900X and a board like yours. But you have the combination right now so considerations are a bit different.
The Prime's VRM isn't very capable but then you can't overclock that 5900X on A320 either. If it's not getting excessively hot with the way you use it you won't really gain much there. Long processing with all cores on things like video rendering will be where temp will go crazy on the VRM's FET's and capacitors. The problem is you may not be able to easily measure temperature since Asus screws owners by not providing VRM temp sensors on their mid and low-end boards.
An IR thermal gun will help with temp measurements, as does locating a fan with tie straps or brackets to blow on the FET's and cool them off. Temperature there up to 105C in extreme cases (long video rendering for instance) should be OK. AM4 VRM's usually throttle the processor when it's temperature reaches around 115C. Modern FET Tjmax is 125C.
You should be able to get 3600 memory to work on the Prime with a 4th gen Ryzen. That's all that's really necessary since going with higher memory clocks often means de-coupling infinity fabric which results in overall lower performance. That makes 3600 the sweet spot.
I'm not sure you can with A320 but a B550 will definitely allow you to enable PBO2, undervolt with Curve Optimizer and even add up to 200Mhz additional boost clock offset. That can help performance for gaming especially, although it requires equally more capable cooling on the CPU. If you can do that with your A320 you'll just overheat the VRM way sooner, making it throttle the CPU to save itself.
So bottom line is: if your useage is undemanding (gaming typically is not demanding in this sense) with no desire to overclock, even with PBO, or experiment with extreme high memory overclocking you'll not likely gain much if anything for the effort and cost of upgrading to B550. If you do decide to, avoid Asus and go with a board that's higher in value. You can easily get a board with a VRM that's no more capable than what you have by cheaping out too much.