4gb isn't that much used, my PC at idle uses 3gb so its not a very big variation.
try as clean boot and see if that changes anything, that will tell you if it is a start up program eating ram
Download
Process explorer and run it as admin (it comes from Microsoft so its safe)
the default view is tree structure meaning like your task manager screen, it will show what services are under each process but unlike task manager, it shows the ram usage of each part so you can see what is eating your ram
Private bytes = actual ram usage
Working set = Ram + page file usage
This page shows what all the colours and headings mean, link at bottom of it shows how to use it to find problems. You can right click processes and run an av...