Is the connection dropping? Or is the modem rebooting? If the connection is dropping, then going off the readings you are giving, its your ISP. Keep in mind there can be internet outages with perfect upstream and downstream, all those readings are is your connection rate to the ISP.
If the modem is rebooting thats on your side. You can tell its rebooting because lights will be flashing when you lose internet. When it does it check to see if the unit is warm, overheating can make it do that. Something may have shifted in the house, or the heater from winter may be heating it up somehow. Make sure its not buried, lack of airflow may cause it to overheat even if the outside isnt warm. If you can rule out overheating then onto troubleshooting.
This doesnt happen often, but sometimes and ISP will update something on their end that will conflict with a modem running old firmware. If this is the case then when you log into your router there will be a notice at the top of the screen asking if you want to update the software, or firmware. Do so if there is one.
To me, with the hours you say its fine, an when it starts it either sounds like the heater is kicking on cause its cold outside and warming the router up....or your ISP is having a flood of users come on just getting off work. It could be your ISP has some equipment down that isnt allowing them to handle the load as well as normal, so if the first 2 things I said to check dont pan out call tech support and tell them whats happening. Most of the time if you ask for a lvl 2 tech you dont have to go through the annoying "unplug/replug" crap.
If your ISP says it isnt them, that its on your end chances are theyll want to replace your router, an problem solved. If you own it, an they wont take responsibility, then come back and there is other things to try.