You don't crontrol brightness on a TV through HDMI. you control brightness on a TV using the TV remote and the picture settings in the TV menu.
Because it's a TV.
Most TVs nowadays have the ability to remember different picture settings for each of the inputs so when you switch from say your cable box or whatever that is plugged into a different input than the PC to the one that is the PC the picture settings are different if you've adjusted them to your liking for the cable box or whatnot from other inputs. Other inputs that have not been adjusted will be at default settings.
But this also depends on your TV as while most HDTVs can do this not all of them do.