Removing the side panel would be more or less a temporary fix. The inside of the case will likely become more dusty and would really only help if your case has poor ventilation. Looking at it, it doesn't seem to have the best ventilation setup so eventually a new/different case may be in order. That one is somewhat limited by a single 120mm fan up front and an 80mm fan in the rear. Most newer cases use at least 120mm fans if not 140mm which move more air volume.
Another factor is going to be your ambient room temps. If your room temps are a bit higher (28-30c +) you're going to want as much ventilation as you can get. All pc cooling is reliant on air and ambient room temps, if the air in your pc room is already warm then heat will only build up faster from components (mainly cpu and gpu). All the fans in the world blowing on max with 30c + air to start with can only cool so much.
I can't say for sure since I don't have the case in front of me but looking at the details and photos on newegg, it appears that the side vent for the graphics card doesn't have any fan mount provisions. It looks like it's just a screen, no holes for screwing fans there.
It may be worth checking out black friday/cyber monday deals since they're coming up. So long as you're not completely against the way it looks (looks are a factor and personal taste when it comes to cases), there's the coolermaster haf912. Newegg has it with a mail in rebate which drops the price to $40. Has bottom/front dust filters, comes with a 120mm fan in front and rear exhaust with options to add a side fan for the graphics card, another 120mm intake in front and up to 2 120mm fans on the top. It's not the newest case around and definitely not the only one out there but not bad for the price.