So yeah, that's normal with a stock cooler but I'd get a better cooler to reduce the CPU fan noise especially under load.
One of the best budget coolers is the Cryorig H7 (see pcpartpicker, Newegg etc. approx. $35USD).
I've used the Noctua NH-U12S or similar in some builds but they are $60+.
(and make sure the motherboard fan software is installed and the fan profile is optimized. roughly keep CPU below 75degC under load. I'm at 300RPM on the Noctua cooler in idle so that's DEAD SILENT then and barely noticeable under heavy load)