I have the same, the input lag actually is also when does not play but when you play you see every moment a bit slowly cause the focus.
Actually the input lag can increse or decrase due to some factors:
peripherial, keyboar or mouse that have high latency
programs, some programs actually have more input lag cause they are not optimized for that
system lag, some times peripherical signals are not processed fast as other thing(expecially when the pc is in full load), the signial have to be processed by the cpu and in game some input can be lost.
All you have to do is check the latency of your peripheric , see if in the same application there is this problem and while play(if you can) see the actual input lag.
Some mode for detect or reduce input lag are nvidia relex(for nvidia gpu owner).
Nvidia reflex, if you have a monitor that support it can detect the input latency while in some games reduce the actual inpulag, I use it on fortnite and you can feel the difference(if used with boost can take some fps like 2,cause the gpu have to do some things ), input lag is also reduced by high fps and in online games by low ping but there is not your fault but is actual high latency for the ping.