The Xeon E5620 features the Westmere architecture that was also used in certain Core i7 chips.
Most comperable would be the i7-970 that features 6 cores and a 3.2 clock without turbo and performs around an i5-2400, i5-4430 or i3-6100
So it could be enough for the Vive to run games. The question is will gaming on the Vive benefit from the 8 cores and 16 threads. If not, and your rig is not overclocked you might run into issues. If all the threads are used properly you would have a still very powerful gaming rig even today.