[SOLVED] U.2 To PCIE

[TryingToVirtualizeStuff]

Hello,

I had a question regarding U.2 to PCIE conversion.

I wish to purchase the following motherboard: Asus Pro-WS-WRX80E-SAGE-SE-WIFI - https://www.asus.com/us/Motherboards-Components/Motherboards/Workstation/Pro-WS-WRX80E-SAGE-SE-WIFI/

...which has 2x U.2 slots and was wondering, is it possible to convert each U.2 to a PCIE 3.0+ 4x?

If so, should this be just a plug and play affair?

Thank you,
Nelson

 

[USAFRet]

That already has 3x M.2 ports.

What drives were you planning on using with the U.2 spaces?

 

[TryingToVirtualizeStuff]

That already has 3x M.2 ports.

What drives were you planning on using with the U.2 spaces?

Hello USAFRet,

The problem with the M.2 ports, is that there may not be enough space for everything to fit.

The plan was to build out the following system (the chassis supports 11 expansion slots - all PCIE cards are single slot except for the GPU which is dual slot - so all 11 expansion slots would be necessary - which will be an extremely tight fit):

https://pasteboard.co/ztXdnPvMh3PQ.bmp

While Googling for solutions, I found the following:

https://www.ioi.com.tw/products/pro...=116&DeviceID=3035&HostID=2082&ProdID=1160005

...which would at least help condense things somewhat.

And then found the following:

https://www.ioi.com.tw/products/proddetail.aspx?AppID=1008&CatID=116&HostID=2082&ProdID=1160002

...which should allow me to fit everything in (as things will be tight with the M.2 to PCIE riser cards).

Any thoughts?

Thank you,
Nelson

 

[USAFRet]

An NVMe drive in a M.2 port takes up pretty much zero space.


We'd need a full list of everything that is going in this system.

 

[TryingToVirtualizeStuff]

An NVMe drive in a M.2 port takes up pretty much zero space.


We'd need a full list of everything that is going in this system.

Hello USAFRet,

It's not an NVME drive that is being inserted into the M.2 port.

It's an PCIE riser (3 of them to be correct) that are inserting into the 3, M.2 ports.

The riser cards are wide enough, that I'm not sure you can fit 3 of them next to one another to utilize the remaining 3 expansion slots on the chassis.

Thanks,
Nelson

 

[USAFRet]

It's not an NVME drive that is being inserted into the M.2 port.

It's an PCIE riser (3 of them to be correct) that are inserting into the 3, M.2 ports.

And I will ask....Why?

 

[TryingToVirtualizeStuff]

And I will ask....Why?

Why the reason for all the PCIE risers?

Simply to hit a density count requested by my company.

Building out two servers (even with a 32 core CPU) is way too expensive for my company.

So everything needs to be pushed into a single server (which makes the cost of everything acceptable).

Regards,
Nelson

 

[Rogue Leader]

Why the reason for all the PCIE risers?

Simply to hit a density count requested by my company.

Building out two servers (even with a 32 core CPU) is way too expensive for my company.

So everything needs to be pushed into a single server (which makes the cost of everything acceptable).

Regards,
Nelson

What in the world is this server supposed to do?

You have capture cards and SDI/ASI card in a Server? With a crap ton of storage. Jamming all this hardware doing all sorts of stuff is just one big hot mess.

Those types of cards belong in workstations for the people who are working on them to use.

Have you built something like this before?

 

[USAFRet]

Simply to hit a density count requested by my company.

The wrong people are setting the requirements for this.

 

[TryingToVirtualizeStuff]

What in the world is this server supposed to do?

You have capture cards and SDI/ASI card in a Server? With a crap ton of storage. Jamming all this hardware doing all sorts of stuff is just one big hot mess.

Those types of cards belong in workstations for the people who are working on them to use.

Have you built something like this before?

The server is a QA box for ASI, SDI, SMPTE 2022-2, SMPTE 2022-6 and SMPTE-2110 (analysis, playout, capture and PSNR testing).

Since we're dealing with up to 8K SDI (48 Gbps), we need a lot of storage + extremely fast storage.

The box is complicated, that's true, but unfortunately in the broadcast/IP domain, things tend to be.

Regards,
Nelson

 

[TryingToVirtualizeStuff]

The wrong people are setting the requirements for this.

Hello USAFRet,

You're absolutely right about that,

But when you work for a small company, with a tight budget, in an expensive domain (such as TV Broadcast), it is what it is.

If I worked for a Defense contractor, this wouldn't be a problem (I could probably order a SuperMicro server for each thing being tested), but that's not that type of company that I work for.

Regards,
Nelson

 

[Rogue Leader]

The server is a QA box for ASI, SDI, SMPTE 2022-2, SMPTE 2022-6 and SMPTE-2110 (analysis, playout, capture and PSNR testing).

Since we're dealing with up to 8K SDI (48 Gbps), we need a lot of storage + extremely fast storage.

The box is complicated, that's true, but unfortunately in the broadcast/IP domain, things tend to be.

Regards,
Nelson

The storage makes sense of course, I'm more concerned about the mix of hardware.

Do you have an intention for multiple people to be accessing it?

I understand what you're doing (not thoroughly as I'm not in the business) but from what I do know you're trying to do WAY too much with one thing.

 

[Rogue Leader]

Hello USAFRet,

You're absolutely right about that,

But when you work for a small company, with a tight budget, in an expensive domain (such as TV Broadcast), it is what it is.

If I worked for a Defense contractor, this wouldn't be a problem (I could probably order a SuperMicro server for each thing being tested), but that's not that type of company that I work for.

Regards,
Nelson

Understandable, but their lack of understanding (and budget) is going to end up in you throwing a whole bunch of money into a machine that isn't going to do the tasks you want it to do well if at all.

 

[TryingToVirtualizeStuff]

The storage makes sense of course, I'm more concerned about the mix of hardware.

Do you have an intention for multiple people to be accessing it?

I understand what you're doing (not thoroughly as I'm not in the business) but from what I do know you're trying to do WAY too much with one thing.

Hello Rogue Leader,

The box will be shared amongst 6 to 7 Engineers.

Each Engineer will likely use a subset of the hardware on the box.

As for virtualization, we're looking into something like Citrix Virtual Apps (or Parallels Virtual Apps), but those software solutions can be expensive.

Worst-case scenario, one person logs into the box at a time (which isn't too bad, as a lot of what will be running on the box is analysis software (constantly monitoring incoming signals) which we'd need to look at when something goes wrong with the hardware/software being tested by the box - so it's rare that someone would need to tie up the box all day - (usage per Engineer would likely be between 30 minutes to an hour every day)).

I know the box is extremely complicated, but my company doesn't have the budget for multiple servers.

Regards,
Nelson

 

[TryingToVirtualizeStuff]

Understandable, but their lack of understanding (and budget) is going to end up in you throwing a whole bunch of money into a machine that isn't going to do the tasks you want it to do well if at all.

Hello Rogue Leader,

Technically, everything should work (as far as CPU, RAM, storage, and PCIE bandwidth goes).

As far as how easy the system is to use for everyone, in that case, yes, it may not be a smooth/easy experience for everyone.

But I suppose compromises must be made, because it's either this, or continue to use old outdated hardware (or extremely expensive specialized units which do a only a fraction of what this box does).

Regards,
Nelson

 

[Rogue Leader]

Hello Rogue Leader,

Technically, everything should work (as far as CPU, RAM, storage, and PCIE bandwidth goes).

As far as how easy the system is to use for everyone, in that case, yes, it may not be a smooth/easy experience for everyone.

But I suppose compromises must be made, because it's either this, or continue to use old outdated hardware (or extremely expensive specialized units which do a only a fraction of what this box does).

Regards,
Nelson

Oh I'm sure it will work and function. I think you're going to have massive problems trying to virtualize this.

I guess my point was I think you're going to spend more time fighting it, and then its not going to finish the tasks you need it to finish.

I feel like your effort is better placed into trying to adjust the budget or expectations.

 

[TryingToVirtualizeStuff]

Oh I'm sure it will work and function. I think you're going to have massive problems trying to virtualize this.

I guess my point was I think you're going to spend more time fighting it, and then its not going to finish the tasks you need it to finish.

I feel like your effort is better placed into trying to adjust the budget or expectations.

Hello Rogue Leader,

If at the very least things work (whether they can be virtualized or not), that's good enough for me.

My company may not like it - but they need to hire someone to do this, and not just push it off to me (a QA manager who had some extra time to look into this).

This is the problem with working at a small company, where everyone wears 4 or 5 hats.

Thanks again for the advice and help,
Nelson

 

[Rogue Leader]

Hello Rogue Leader,

If at the very least things work (whether they can be virtualized or not), that's good enough for me.

My company may not like it - but they need to hire someone to do this, and not just push it off to me (a QA manager who had some extra time to look into this).

This is the problem with working at a small company, where everyone wears 4 or 5 hats.

Thanks again for the advice and help,
Nelson

100% agree.

My goal here is to prevent you from spending your limited budget on something that won't work. I am pretty sure this won't work. I mean it will turn on, and you probably can get some of the devices to run. But you're going to have conflicts and issues.

Like I said, turn around and tell them this isn't going to work the way they think it will, and advise them that they will be burning the budget on something that will end up useless, unless they spend even more money to fix it.

Do it right the first time, even if it costs a little more and takes a little more time. Otherwise it will cost a lot more and take much more time.

 

[TryingToVirtualizeStuff]

100% agree.

My goal here is to prevent you from spending your limited budget on something that won't work. I am pretty sure this won't work. I mean it will turn on, and you probably can get some of the devices to run. But you're going to have conflicts and issues.

Like I said, turn around and tell them this isn't going to work the way they think it will, and advise them that they will be burning the budget on something that will end up useless, unless they spend even more money to fix it.

Do it right the first time, even if it costs a little more and takes a little more time. Otherwise it will cost a lot more and take much more time.

Hello Rogue Leader,

Just out of curiosity, when you say the following:

" I mean it will turn on, and you probably can get some of the devices to run. But you're going to have conflicts and issues."

What do you mean?

The reason I ask is that I/O-wise, there seems to be enough of everything. The same applies to CPU, RAM, storage, etc.

When you say there could be conflicts, what sort of conflicts do you mean?

Would these be hardware conflicts, or more like software conflicts?

Just curious, and thank you once again.

Nelson

 

[Ralston18]

Book recommendation:

https://www.amazon.com/Cybersecurity-Leadership-Powering-Modern-Organization/dp/1502312115

Good read for anyone getting pulled into the IT world.

No matter what discipline and company size.

 

[Rogue Leader]

Hello Rogue Leader,

Just out of curiosity, when you say the following:

" I mean it will turn on, and you probably can get some of the devices to run. But you're going to have conflicts and issues."

What do you mean?

The reason I ask is that I/O-wise, there seems to be enough of everything. The same applies to CPU, RAM, storage, etc.

When you say there could be conflicts, what sort of conflicts do you mean?

Would these be hardware conflicts, or more like software conflicts?

Just curious, and thank you once again.

Nelson

Despite having enough PCIe lanes for you to populate most everything I believe you will run into both software and hardware conflicts with that many disparate pieces of hardware.

Of course I can't say this for sure because I've never built something this complicated. But I can tell you piling that many pieces of high end hardware into a small space you will run into heat issues, and conflict issues with different devices that do similar tasks fighting for the task provided. Or multiple devices attempting to utilize the same piece of hardware on the system side.

Boards like this with many slots and CPUs work much better as a render farm, i.e. 4+ GPUs working towards the same project. The various softwares and drivers for all these devices are going to fight. And hey maybe there is a way to figure this out and make it work, but, no offense, you may not be the guy to do it, by your own admission.

I'll say and easy way to test this theory is at least look at some outfits out there that build the type of system(s) you need. I'd bet money you will find none of these pros are stuffing that much disparate hardware into 1 system.