It likely is something wrong with the connection coming to your house.
Since if you call the ISP they will attempt to blame everything else first best to have tested it before you call them. I would take a PC and plug it directly into the modem. In many cases you will have to power cycle the modem and then plug your pc into it after the modem is booted. Modems tend to lock onto the first device they see and not change so it would only talk to your router until it is rebooted. Open a CMD window and leave a constant ping run to 8.8.8.8 or 1.1.1.1. This is so you can show the ISP that you are getting packet loss.
It is strange that is only has issue during those hours. It will make it harder for the ISP to test. You likely should call the ISP during that time frame or they will test from their office and see no problem. If the ISP offers to come out try to get them to come out during that time frame also. At least you have some part of the day where the ISP will be working, it is much harder when it is only when they are closed.
Hard to say what it would be. Mostly when it is time of day issue it is other people traffic causing issues for you. The internet connection to your house is likely shared with many of your neighbors. It is much more common for it to be a problem say when people get home from work (before people worked from home so much).
Maybe it is weather/tempature related if it get cold at night. In any case you can do nothing about this stuff. It is not likely in your house and hooking directly to the modem will eliminate most strange possibilities.