1: The XP mode on 7 doesn't support HW acceleration
2: Compatability modes do nothing 99% of the time; they just call older versions of .dll files to try and improve compatability, but oftentimes the problem lies elsewhere (Changes in Video Card Drivers, low level windows timings, expecting a DOS layer, etc).
The surest way would be to keep a 98SE/ME virtual OS handy (provided you can get GPU drivers for your card...)