Use Task Manager or Resource Monitor to observe your system's performance.
Open one tool or the other and watch for a few minutes. Slide the window to one side and leave open.
Watch while you work or game as normal. Especially in the 3-4 hour timeframe.
My thought is that some app may be launching in order to phone home and/or update. Or some backup process or AV scan may be launching.
Do you have access to your router's logs? The router's logs, if available and enabled, may be capturing some reason for the slow downs.