What are the dimensions of the psu that you have?
Is there a make/model on it?
In your kind of case, it will be a non standard psu that may not have an equal sixed replacement with adequate power for your mew motherboard.
A decent psu will have both a 4 and an 8 pin atx cpu power connector that you need.
The problem is that since the 24 pin psu motherboard connector has no place to plug into the motherboard, there is no way to get the power on signal from the motherboard to start up the psu.
I suppose you could permanently short pin 15(green) on the 24 pin connector with a ground to get the psu started.
You could then use the on/off switch on the back of the psu to turn it in or off.
If your case does not allow a standard ATX psu, you could place the psu on top.
My suggestion is to also buy a standard cheap case, 450w atx psu, and and motherboard for your processor
Have you measured the height of your new card? would it even fit in your case?