It depends what you want the new PC for.
If you predominantly game on PS4, and think that's likely to be the case with/without a new PC, then the VR headset makes the most sense.
If you have other tasks you want to perform that the PS4 can't handle (or handle well), then the PC might make the most sense.
I can't make that decision for you. If it were me, I'd opt for the PC - If/when I want to explore VR, I want it to be a bit more 'advanced' than it is today - and on higher-end hardware than the PS4 offers.