When you split a headphone signal among 5 headphones, the energy that was originally supposed to drive one set of headphones is split 5-ways. One of the consequences is that the volume is lower (each headphone is getting less energy). Another complication is that not all the headphones may have the same impedance, so some headphones may take a greater share of that energy than others.
I see a post was added while I typed. That BEHRINGER MICROAMP looks like a great option with independent volume and all that good stuff.
This could depend on how the splitter works.
If you have a splitter that is parallel you could put quite a load on the sound cards output.
5 pairs of 32 ohm headphones in parallel would look like a 6.4ohm load(this is like driving a home speaker).
Not all cards have the same type of output, but some have a resistor inline with that output so at lower resistance ratings(heavier load) the voltage will also sag making things even worse.
Now to make things more interesting if the headphones are in series, you may have very high resistance(easy load). 5 pairs of 32 ohm headphones will look like a 160ohm speaker. because the load is so light, you need more voltage to drive it. Lets say your computer sound card has 2 volts to use 160 ohms of headphones would give you 0.025watts compared to the 0.125 for 32 ohms(that parallel combination above would try to use 0.625 watts). This is not harmful to the sound card, but just not very loud since all the headphones will share this voltage.
The solution could be something like a headphone amp(or even one per headset as this would allow each users to control the volume) or use a small power amplifier(lots of older amplifiers had headphone jacks that could drive speakers.).
Remember amplifiers with headphones in parallel will be more than capable to cause hearing and headphone damage.
Series connection may be a bit safer. An amp that can put 50 watts into an 8 ohm speaker will only put 2.5 watts into a 160 ohm headset splitter, BUT that will kill your headphones and hearing so you have to turn it down and adjust it after.