Are you using an amd apu or intel cpu and using the integrated graphics? That might explain it. As others suggested, there's a 4gb memory limit with 32bit windows and many times not even that much. It may cap you at 3.7 or 3.8gb.
If you're also using integrated graphics the igpu may be using 1.7gb for video ram since integrated graphics don't have their own video ram like a dedicated gpu does. It shares the system memory. 3.7gb usable minus 1.7gb for integrated graphics and you'd only have 2gb left for windows and programs. You should consider getting a 64bit version of windows to use all the ram.