Yeah GPU utilisation doesn't work like CPU utilisation does. As long as anything is happening on the GPU it counts as utilisation. It is likely due to the architecture, with CPUs time utilisation on a core very closely correlates to actual resource utilisation whereas GPUs are different, likely due to them being split into SM and then individual cores. An SM could be running something 100 % of the time but could only actually be using a single core so whilst the time utilisation of the SM is 100 %, the actual resource utilisation is very low.
I am very surprised they don't have better tools for measuring GPU utilisation though. I know Nvidia has NSight but that is a bit beyond just showing a percentage in task manager.