If your taskbar shows means you're not in true full screen,
Even if you are in windowed mode, but using the whole monitor it shouldn't show up if on auto-hide except if there is a application soliciting it for update like a chat client flashing icon on the taskbar or a system Tray notification.
There is also an intermediate mode which is normal windowed game but frame-less or windowed borderless that could be achieved with some third party apps or sometimes glitches some games when going from full screen to desktop mode using ALT-TAB instead of the usual CTRL-ENTER or ALT-ENTER ( I use it all the time when playing Dirt3)