Given boot times these days, I see no reason to keep it running 24/7. There is some argument for temperature cycling and the negative effects there, but it really comes down to how long you intend to keep it. For myself, I upgrade somewhat regularly, so the longevity of the components don't matter as much. But if you want to keep it going, that just means regular maintenance. Cleaning out dust filters, changing thermal compound every year or two, swapping out fans as they fail. Just generally monitoring temperatures to see if anything is going wrong.
If your computer runs significantly cooler with the side panel off, it means you haven't really achieved proper airflow. In an ideal world you would have enough incoming filtered air and enough exhaust to keep the system cool and without dust getting in (too much) Not that hard to achieve really.
The side panel is there primarily to protect the contents of the computer, so leaving it off isn't wise from that perspective. In many designs it is also necessary for proper airflow. If all the intake fans do is end up pushing air out of the open side panel, they are only effectively cooling the components mounted in the front (Assuming front intake)
I've done both. In the XP days I pretty much left my machine on 24/7 to avoid boot times. Air filters were almost unheard of in computers. The era of side panel fans that blew external air directly into the CPU cooler. Made for pretty dusty computers.
Now I only turn my gaming system on when I need it. Probably not ideal for the pump to have that many 'dry' starts, but everything else should be getting less hours on it. I don't mind swapping out the cooler in a year or two. Actually kind of need to, that thing is getting on 5 or so years old.