Question nVidia/Mellanox ConnectX6+ stuff at a sane price -where and how ?

[Quinn2_Santini3]

I'm looking at getting some faster Ethernet NICs etc and everyone keeps pissing on Intel's E8xxx series as buggy.
Same with Broadcom.

Only other option seems to be nVidia. But their NICs tend to be atrociously expensive, at least outside of datacenter deals.

Is there a trick to getting them for sane price ?
And I don't mean usual e-bay finds for prehistoric Connect-X4/5 etc as they have, if nothing else, slow PCIe interface, but something that uses PCIe4/5 and wouldn't waste precious few PCIe lanes on modern cheap host that one might use at home.

So, how do mere mortals get to those ?

 

[kanewolf]

I'm looking at getting some faster Ethernet NICs etc and everyone keeps pissing on Intel's E8xxx series as buggy.
Same with Broadcom.

Only other option seems to be nVidia. But their NICs tend to be atrociously expensive, at least outside of datacenter deals.

Is there a trick to getting them for sane price ?
And I don't mean usual e-bay finds for prehistoric Connect-X4/5 etc as they have, if nothing else, slow PCIe interface, but something that uses PCIe4/5 and wouldn't waste precious few PCIe lanes on modern cheap host that one might use at home.

So, how do mere mortals get to those ?

What does "faster" mean? 10GE? 25GE? 40GE?? 100GE ??

 

[Quinn2_Santini3]

25GbE, 100GbE and now also 200GbE for me.

 

[kanewolf]

25GbE, 100GbE and now also 200GbE for me.

What hosts do you have that can utilize 100GE ?

 

[Quinn2_Santini3]

Do you need their names, maybe family tree of their owners also ?

 

[cruisetung]

Why do you think a consumer has a "trick" to buy a high end network card at a sane price after all?

Asking what PC host you have to see if it can handle 200Gbe is also a sane question.

 

[kanewolf]

Do you need their names, maybe family tree of their owners also ?

I was just asking what type hosts you have that can handle that much bandwidth. That would be an unusual set of hardware.
A 100GE switch chassis is $5K+ and the optics are as expensive as the chassis. So 100GE NIC capable hosts are not the norm.

 

[Quinn2_Santini3]

But the NIC itself is $400+ new and used ones go for $200.

Cheapest new 100G switch is $500+ and 16-port ones are IIRC $1.500.

On top of that, one might not need a switch as 100G NIC can be split into 4x25G.

SO it can come handy for example, for small star topologies for 25G networking.
One could simply plop 100G NIC into server box, split it 4x and plop up to 4 25G clients onto it.

best thing is that cloents wouldn't have to compete for bandwith.

Hence the interest in new 200G "cheap" gear that is to hit shelves with newest Intel's E830 series.

Even if one doesn't need 200G networking, Intel's 200G E830 can be split 8-way x25G.

 

[Quinn2_Santini3]

Asking what PC host you have to see if it can handle 200Gbe is also a sane question.

But off-topic.

Also, answer would be simple - pretty much any host that has the PCIe bandwidth and at least RAM bandwidth.

Also, full saturation capability is not always needed. If one bifurcates the NIC for N x50G or M x25G clients and connect them directly through DAC-splitters to avoid having a switch, getting max cummulative bandwidth might not be a priority or even important.

As these things support RDMA, so there is not much CPU load.
All that might be really needed is RAM bandwith.

 

[Quinn2_Santini3]

A 100GE switch chassis is $5K+ and the optics are as expensive as the chassis. So 100GE NIC capable hosts are not the norm.

In case you didn't know about new cheap/er options:

• MikroTik CRS504-4XQ-IN A 4x 100GbE Switch at Under 45W
• MikroTik CRS520-4XS-16XQ-RM 16-port 100GbE
• MikroTik CRS518-16XS-2XQ-RM Review Cheaper 25GbE and 100GbE Switching

 

[cruisetung]

Intel 200G E830 won't be available until later this year

Intel E830 for 25GbE to 200GbE and E610 for 10GbE and 2.5GbE NICs Launched

Intel has new dual port 2.5GbE and 10Gbase-T NICs with the Intel E610 series as well as 25GbE to 200GbE in the Intel E830 series

www.servethehome.com

On the Intel E830, the first slide Intel showed said it was a 200GbE controller and much faster than the prvious generation. The actual first part is going to be the Intel E830-XXVDA2 which is a dual 25GbE part only. Intel says other models will follow, but at launch the E830 should only offer about half the bandwidth of the prior-generation Intel E810 not twice the bandwidth. Intel says additional configurations for the E830 will arrive later in 2025 so hopefully those bring 200GbE speeds.

I asked Grok the question that you have
https://grok.com/chat/1616425e-512c-4fcb-ab88-a43b35c279db

Try other AI platforms as well

 

[Quinn2_Santini3]

Intel 200G E830 won't be available until later this year

Intel E830 for 25GbE to 200GbE and E610 for 10GbE and 2.5GbE NICs Launched

Intel has new dual port 2.5GbE and 10Gbase-T NICs with the Intel E610 series as well as 25GbE to 200GbE in the Intel E830 series

www.servethehome.com

I asked Grok the question that you have
https://grok.com/chat/1616425e-512c-4fcb-ab88-a43b35c279db

Try other AI platforms as well

Link not shared correctly
If this was a link someone shared with you, please ask the sender to explicitly share the conversation.

 

[cruisetung]

See if this works

https://grok.com/share/bGVnYWN5_88052a59-b9b0-430b-a3bc-65fdbe9bab5e