Question Asus Z270-P doesn't POST with two GPUs installed ?

Toggle sidebar Toggle sidebar
Subcode

Subcode

Jul 2, 2023
12
1
15
Hi,

I'm having horrible issues trying to get my two P40s working on my Z270-P motherboard.

I have above 4G decoding on, if i dont have it on i get a "PCI resource allocation error".
But when i have it enabled and try to boot the system with 2 GPUs it does not POST.
Nothing is shown on screen at all. And i cannot get into the BIOS.

I have the display output set to onboard so its not trying to show it on the P40s (they dont have video out). And i can see that it doesnt boot into the system because the webui never comes up.

I've tried everything, there is nothing else attached to the system.
If i only use 1 card there is no problem at all with 4g decoding on. If i disable 4G decoding it boots fine with 2 cards, but the drivers wont see the GPUs, they show up but arent usable.

I'm using UEFI cause i read somewhere that matters, but it doesnt help.

I do notice that the fans spin up to the set curve, so its doing something, it just doesnt get past there.

Anyone have any ideas? I really dont get why it doesnt atleast post and give me an error message.

System:
CPU: 7700k
Mobo: Asus Prime Z270-P
GPU: 2x Nvidia Tesla P40
 
I'm assuming that the BIOS on the motherboard has been updated. I'm pretty sure you're right about needing the "Above 4G Decoding" for the P40s to work properly.

Does a running a single card work when it's put in slot 2?

I'm just wondering if it is some sort of weird problem being caused by that slot being off the chipset. The manual is extremely unclear, but it looks like the M.2 also run off of the chipset so if you have any such SSDs there could be an issue.
 
thestryker said:
I'm assuming that the BIOS on the motherboard has been updated. I'm pretty sure you're right about needing the "Above 4G Decoding" for the P40s to work properly.

Does a running a single card work when it's put in slot 2?

I'm just wondering if it is some sort of weird problem being caused by that slot being off the chipset. The manual is extremely unclear, but it looks like the M.2 also run off of the chipset so if you have any such SSDs there could be an issue.
Click to expand...
Yeah one card works fine in slot 2. i dont have an m.2 there. I did have an m.2 slot in in the top slot but i removed that and tried without one. That didnt seem to change anything.

Im honestly not sure if a single card would work in slot 1 since that is the on that is easiest to remove.
I will have a look in a bit and edit this post when i know the result.

--edit
A single card in slot 1 or 2 works fine, if i use both it does not.
 
Last edited:
I'm at a loss here because it's almost acting like when you have the two cards in and "Above 4G Decoding" set it's ignoring your IGP selection. If there was going to be an issue slots wise it would have been with the second one.

I'm sure your graphics are set fine, but just looking through the manual for the board this is what I'd try.

I'm sure these two are already set this way:
Primary Display [CPU Graphics]
iGPU Multi-Monitor [Disabled]

Change this one if it's set to enabled, I doubt this is the issue, but it won't hurt anything:
RC6(Render Standby) [Disabled]

It doesn't really matter what you put in this one, but just raise it from whatever it is to see if that makes any difference:
DVMT Pre-Allocated [128M]

None of the other graphics settings I saw in the manual would have any impact on allocation or output as they're all centered around voltage controls and overclocking. You could potentially try to set PCIe manually to 3.0 including the DMI link to the chipset, but I doubt this will have any impact either.
 
I've tried it but that unfortunately also didnt help.
Its a complete mindf*ck.. The fan curve gets set correctly but just no ASUS logo or bios at all.
Could it still be something with the cards?
They work individually but is it possible something doesnt work when theyre both used?
Or maybe the cards are just too powerful for the motherboard? They both have 24GB VRAM for a total of 48GB. Could that be too much?
If for some reason itd be a power issue the mobo wouldnt start / reset aswel right?

I have an old Bitcoin mining mobo (Made to run max 12 cards) i got from a friend. Bought a 6 euro riser for it to see if that will run with the 2 cards. If that cant do it either then im going to asume its something with the cards. But i wish the BIOS would give me some kind of error log to see why it wont POST. Its so frustrating.
 
While I can't think of any reason why they shouldn't work they certainly were never intended for consumer platforms. I'd looked to see if anyone else had tried that setup with your platform, but all I could find was single card.

It really sounds like the system might still be booting, but just not giving you proper video out. I didn't see anything in the manual regarding debug so I'm assuming all it has is speaker beeps.
 
thestryker said:
While I can't think of any reason why they shouldn't work they certainly were never intended for consumer platforms. I'd looked to see if anyone else had tried that setup with your platform, but all I could find was single card.

It really sounds like the system might still be booting, but just not giving you proper video out. I didn't see anything in the manual regarding debug so I'm assuming all it has is speaker beeps.
Click to expand...
Ive tried leaving it on for a while but the webui never pops up, The thing thats also interresting is that when its booted or after post i usually have to hold the power button to get it to turn off. But in this state a single press instantly turns it off.
It actually doesnt have a speaker connected, but i can look around to see if i can find one to attach (the piezo kind)
 
Subcode said:
Ive tried leaving it on for a while but the webui never pops up, The thing thats also interresting is that when its booted or after post i usually have to hold the power button to get it to turn off. But in this state a single press instantly turns it off.
It actually doesnt have a speaker connected, but i can look around to see if i can find one to attach (the piezo kind)
Click to expand...
It's definitely not booting then if it powers right off, but that also doesn't help with where it's getting hung up. Hopefully hooking up a speaker will give you some beep codes, but I'd bet it will be video at this point.
 
thestryker said:
It's definitely not booting then if it powers right off, but that also doesn't help with where it's getting hung up. Hopefully hooking up a speaker will give you some beep codes, but I'd bet it will be video at this point.
Click to expand...
You mean a problem with the videocards? Or just the mobo not being able to run them.
I've tried looking for a speaker but it has some weird 4-pin kind im not familiar with and didnt seem to come with it. Cant find any that would work either :/ Would have to order one if you think thats my best bet.

Asuming its the motherboard not allowing this to work, would you have a recommendation for a system that would allow it? I'd prefer to not go with a super loud server but if thats the only possibility i'll atleast look into it.

edit:
I tried with a ASUS Z270-A which has qdebug lights. This gives me a solid white VGA light which would indicate a faulty GPU or some other fault with the GPU. Not sure if this would also be the case of incompatibility.
 
Last edited:
Assuming this is accurate you have your problem:

Reddit - Dive into anything

www.reddit.com www.reddit.com

You'd need a motherboard which split the lanes from the CPU and supported resizable bar on multiple. Though that wouldn't explain why it booted in the first or second slot but not both which is why I'm just not sure.

It doesn't particularly make sense to me that the GPUs would be bad but only in dual configuration.

edit: if it is platform related the only things I could think of looking at would be x299 Intel, or looking up machines the P40s were available in and getting whatever chipset that runs. Threadripper may also work, but I know the resizable bar support isn't necessarily the best.
 
Last edited:
thestryker said:
Assuming this is accurate you have your problem:

Reddit - Dive into anything

www.reddit.com www.reddit.com

You'd need a motherboard which split the lanes from the CPU and supported resizable bar on multiple. Though that wouldn't explain why it booted in the first or second slot but not both which is why I'm just not sure.

It doesn't particularly make sense to me that the GPUs would be bad but only in dual configuration.

edit: if it is platform related the only things I could think of looking at would be x299 Intel, or looking up machines the P40s were available in and getting whatever chipset that runs. Threadripper may also work, but I know the resizable bar support isn't necessarily the best.
Click to expand...
Someone on reddit said it might be that the P40 requires 16 lanes, and my cpu/mobo only has 16. So i would need something that atleast has 32 lanes and can devide them up between the 2 pci slots.
Any ideas?
 
Subcode said:
Someone on reddit said it might be that the P40 requires 16 lanes, and my cpu/mobo only has 16. So i would need something that atleast has 32 lanes and can devide them up between the 2 pci slots.
Any ideas?
Click to expand...
Your second slot only provides 4 no matter what so I really don't think this is the issue since you'd said a singular card worked in it.
 
thestryker said:
Your second slot only provides 4 no matter what so I really don't think this is the issue since you'd said a singular card worked in it.
Click to expand...
Ah this is true i guess. But not possible it auto reassigns the 16 it has to that slot? I thought i read something like that in the manual. It said something along the lines of 1 card is 16 lanes, But didnt specify which slot it was.
 
Subcode said:
Ah this is true i guess. But not possible it auto reassigns the 16 it has to that slot? I thought i read something like that in the manual. It said something along the lines of 1 card is 16 lanes, But didnt specify which slot it was.
Click to expand...
No that slot is from the chipset on your board.

Subcode said:

Hardware Reference Configuration Guide — MapD 3.2.0 documentation

docs.omnisci.com docs.omnisci.com

Found an example system here with 2 p40s, its full on server hardware so very loud. But im trying to figure out what the difference would be and why this would work but my setup wouldnt.
Click to expand...
Most of those will be server platforms which were designed for that sort of thing in advance. It could be a matter of the addressing or even maybe that the second slot in your system is off the chipset and maybe the "Above 4G decoding" doesn't work on both. It's really hard to narrow down what specifically the problem is when you don't have another platform to test the cards with unfortunately.

Saw this citing x570 AMD desktop platform so maybe it is resizable bar support:

Reddit - Dive into anything

www.reddit.com www.reddit.com
 
thestryker said:
No that slot is from the chipset on your board.


Most of those will be server platforms which were designed for that sort of thing in advance. It could be a matter of the addressing or even maybe that the second slot in your system is off the chipset and maybe the "Above 4G decoding" doesn't work on both. It's really hard to narrow down what specifically the problem is when you don't have another platform to test the cards with unfortunately.

Saw this citing x570 AMD desktop platform so maybe it is resizable bar support:

Reddit - Dive into anything

www.reddit.com www.reddit.com
Click to expand...
Awesome, that looks like what im looking for.
I guess i'll try that then!
So much better than trying to find a loud server and if its confirmed to be working i can just buy those parts form the store. If it then doesnt i can return them without a problem.
 
I've ordered the same system for testing, should arrive within 2 days so i will update the post if it turns out to be successful.
 
You must log in or register to reply here.

TRENDING THREADS

Latest posts

Moderators online

Share this page

COMPANY
RESOURCES
FOLLOW
Top Bottom