So, I have a Logitech k120 keyboard and I built my own PC.
What this means is that I don't have any special keyboard software beyond the standard windows driver. I kept searching for all the other settings that someone was suggesting. The widcomm registry hack, the logitch appdata config file. None of those were present for me. Someone had mentioned bluetooth in my searches for an answer.
I had installed drivers for a crappy usb bluetooth dongle that I had gotten. It was called csr bluetooth or something like that. After uninstalling the bluetooth driver the caps lock num lock etc popup problem disappeared.