I would say not worth it. I have 4GB in my HTPC. I am using an A6 APU so I have 1GB reserved for Graphics and I have yet to see my HTPC have less then 1 GB free space. Of course it depends how you are using it. I use mine for Viewing internet, video and TV only (outside of recording OTA TV).
For how you use your HTPC what feels slow / sluggish about it?
If nothing, then no reason to waste money on RAM at this point. RAM really only limits performance if you don't have enough.
If something seems slow it is much more likely to be either your network or HDD for viewing, and if you are doing any file conversions then your CPU.