An OS reads/writes data by reading strings of bits, and the number of consecutive bits that is read by the OS is the limit to how much storage (memory) an OS can address at any one time. [An OS refers to memory by location, so each location must have a unique numbered value. What locations are in use by the OS, and what data is stored at those locations, are placed into an Address Table].
However, there are only so many unique combinations that can be obtained by a string of 32-bit values. [EG: 00000000000000000000000000000000, 00000000000000000000000000000001, 00000000000000000000000000000010, 00000000000000000000000000000011, 00000000000000000000000000000100, 00000000000000000000000000000101, 00000000000000000000000000000111, etc] That numerical limit an OS can count to can be expressed as:
X^n
Where X is the number of combinations for a bit value (0 or 1, so X would always be 2 in this case), and n refers to the number of conecutive bits an OS reads (typically 32 or 64).
As such, a 32-bit OS can read exactly 2^32, or 4,294,967,296 unique memory locations (0 through 4,294,967,295), or 4GB. In otherwords, using a 32-bit string of numbers, there are exactly 4,294,967,296 possible unique combinations of 0 or 1 values.
The reason why a 3GB is incorrectly attributed as the 32bit is because certain HW (motherboard, GPU, etc) eat up RAM before any windows processes even load, which takes a chunk out of the 4GB limit.
There are other ways to expand the limit besides moving to a higher bit OS; PAE is a mechanism which uses multiple address tables (which are used to keep count of what memory locations are currently in use, and what they hold). As more tables are used, each additonal table adds another 4GB of usable RAM (PAE maxes out at 36-bits, or 64GB), but because the size of an individual table is still 4GB, and because no single process can occupy more then one table at a time, even though 64GB total can be used, no single process can use more then the total space of a single address table (4GB) at any one time, so PAE is an inferior mechanism compared to a higher bit OS.