This usually happens when you subscribe to a thread that gets removed. If you go to your Tracked Threads, there's a link called Mark all as read (across from the Unread answers only button) and clicking that should remove the notification.
The other time this happens for me, it's because I signed in using Incognito or Private Mode in my browser. Then it always shows the number of notifications at the time I logged in, regardless of whether I go through them or not. It's probably related to blocking tracking information. Sessions also expire strangely, where it will show me logged in on some pages but not in others, until I attempt to access my profile and it says welcome back and asks to confirm my login.