The base starting point for all cooling is the ambient temperature.
First, check what your cpu idle temperature is.
At idle, I would expect to see 10-15c. over ambient, regardless of the cooler type.
Your side fan is a good way to insure that your cooler gets the fresh air that is needed to do the job.
If you have Air conditioning in your room, that is good.
If not, heat will build up and something as simple as a window exhaust fan will get the heat out of your room.
Heat is not good for any components, but a cpu and graphics card are designed to tolerate a certain amount of heat.
You can get a bit paranoid about this.
If you found that a house fan helps, then perhaps you could consider a better cooling case.