The 8400 GS is an ancient card, and as you say it uses a different interface, it uses AGP, which is now obsolete; the current interface is PCIe. And no, there are no adapters AGP-PCIe. As your motherboard has a 16x PCIe slot, you should get any PCIe card (which is any GPU from the last 10 years). But if you have a 8400 GS, you won't be able to use it on that PC.
You can use a card with any amount of memory (256MB, 512MB, 1GB or even more, there's no limitation).
EDIT: my bad, i misread, your card is really PCIe, not AGP. So you should be able to use it on the 16x PCIe port in your motherboard. The "different connector" you see on the motherboard might be the PCI slot? ("PCI" and "PCIe" are different: the PCIe port on your motherboard is the long one that is closer to the processor). The PCIe slot hasn't changed since its creation, so it should take your card, however old it might be. But note that any modern card is far, far better than the 8400GS.