Yes indeed, it's because of the capacitors in the PSU which hold residual current even if you unplug it from the wall-socket.
However, you can manually purge the PSU's capacitors after shutdown by unplugging PSU from wall-socket, then keeping PC's front power-on button pressed for around 20 seconds.
Be aware that some USB ports will always remain "live" after shutdown -- this is by design to allow you to charge phones & tablets without turning on the PC.