Fan curve settings are simple, normally the Y-axis (vertical line) represents the % of fan RPM and the X-axis (horizontal line) represents the temperature of the CPU. Simply drag or manually set the curve so the fan percentage has a base speed of whatever you want (50-70% is fine for idle) and then set markers for temps that you know you want the fan to reach higher RPM. It's hard to explain without an example, something like this might help:
View: https://www.youtube.com/watch?v=VgysU-aEBdQ
If you don't have a software application to control your fan on windows you can check to see if your motherboard's BIOS provides a feature to control the CPU fan curve.