Since you have a a minimum of 1ms and a average of 3ms I am assuming you did a ping to your router IP. Since you posted this question on the wireless forum I will also assume that you are using wifi.
This is pretty much what you expect to see on wifi. Your average latency is very good and so it should work well....EXCEPT for online games.
Online games use the delay between packets to keep the client in sync with the server. You get very bad results when the latency is not extremely stable. This is a fundamental design inconsistency between how wifi works and how games network works. This is why you see them say never play online games on wifi. The wifi signal has to be almost perfect for it to work.
The spikes in the wifi are being caused by interfering signal both from inside and outside your house.
Test with a ethernet cable to be 100% sure it is not something else. If it is really a wifi issue there is very little you can do to fix it.