In general the spikes in the ping times mean that data is being held in a buffer someplace. This is generally due to a overloaded connection.
So first I will assume that this is some kind of cable coming into your house....ie you are not running mobile broadband. Any kind radio signal is subject to interference.
I would first try to remove the router and plug directly into the modem. You might have to reboot the modem everytime you change what is plugged into it.
This should eliminate the router as the cause.
I would next run tracert and ping the various hops in the trace. If you have your router installed that will be hop 1 and you should never see more than maybe a spike to 5ms max. Hop2 represents the connection between your house and the ISP.
If you were exceeding your bandwidth you can see spikes at this hop. Note if you were to upload at max rate it will limit your download rates and cause latency spikes. If you see issue here be sure to check that nothing is using bandwidth in your house.
It gets messy after this. Hop2 is shared by you and other neigbors. It is kinda rare to be able to overload this on modern systems. Years a go if a teen in a neighbors house was running torrents it would affect everyone. Now days ISP have huge bandwidth.
Hops farther out can be bandwidth issues between your ISP and other ISP. Nothing you can set or change that will affect this.