Question Intel X540-T1 driver on Windows 10 Pro 22H2 ?

[silvexphoto]

I have Windows 10 Pro 22H2 19045.2913 installed on a SuperMicro X10SRL-F motherboard (latest BIOS) and drivers. The X540 is in a PCI Gen3 x8 slot ( I set the slot to Gen2 in BiOS and no change). The version 28 (Feb 2022) drivers from Intel install and I am able to surf without issues.
The problem is that I can't get more that 1Gb speed on the card. It is connected to a Netgear 10Gb unmanaged switch at 10Gb with a CAT 6A 14ft cable. I have tested two different cables and no change.
Testing it to an iPerf3 local server (CentOS 7) with another X540 installed. I can't get past 2Gbs using the switch and also using a crossover cable (CAT 6A).
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 172.16.200.210, port 3839
[ 5] local 172.16.200.12 port 5201 connected to 172.16.200.210 port 3840
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 117 MBytes 984 Mbits/sec
[ 5] 1.00-2.00 sec 166 MBytes 1.39 Gbits/sec
[ 5] 2.00-3.00 sec 131 MBytes 1.10 Gbits/sec
[ 5] 3.00-4.00 sec 164 MBytes 1.37 Gbits/sec
[ 5] 4.00-5.00 sec 99.4 MBytes 834 Mbits/sec
[ 5] 5.00-6.00 sec 101 MBytes 846 Mbits/sec
[ 5] 6.00-7.00 sec 109 MBytes 917 Mbits/sec
[ 5] 7.00-8.00 sec 105 MBytes 879 Mbits/sec
[ 5] 8.00-9.00 sec 66.9 MBytes 561 Mbits/sec
[ 5] 9.00-10.00 sec 76.2 MBytes 639 Mbits/sec
[ 5] 10.00-10.04 sec 8.19 MBytes 1.57 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.04 sec 1.12 GBytes 955 Mbits/sec receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
My Internet is 2Gbe/2Gbe. The CentOS gets 2Gb/s up/down no issues. Windows under 1Gb/s.
Test from CentOS 7
Speedtest by Ookla
Server: Frontier - Los Angeles, CA (id: 14236)
ISP: Frontier Communications
Idle Latency: 3.11 ms (jitter: 0.77ms, low: 2.30ms, high: 3.44ms)
Download: 2337.37 Mbps (data used: 2.7 GB)
20.96 ms (jitter: 0.98ms, low: 3.58ms, high: 23.99ms)
Upload: 2545.85 Mbps (data used: 2.6 GB)
5.45 ms (jitter: 0.59ms, low: 3.10ms, high: 7.64ms)
Packet Loss: 0.0%
Also PROSET tool does not recognizes the card.
Intel README.txt says that driver has only be test under Windows 10 PRO
Supported Operating Systems
===========================
The drivers in this release have been tested with the following operating
systems (OSs). Additional OSs may function with our drivers but are not tested.
NOTE: Not all devices support all operating systems listed. Refer to the
release notes for detailed OS support information for your device.
Microsoft* Windows Server*, Azure Stack HCI, and Windows*
----------------------------------------------------------
Microsoft Windows Server 2022
Microsoft Windows Server 2019, Version 1903
Microsoft Windows Server 2016
Microsoft Windows Server 2012 R2
Microsoft Windows Server 2012
Microsoft Azure Stack HCI
Microsoft Windows 11 22H2
Microsoft Windows 11 21H2
Microsoft Windows 10 21H2
Microsoft Windows 10, Version 1809

 

[kanewolf]

I have Windows 10 PRO 22H2 19045.2913 installed on a SuperMicro X10SRL-F motherboard (latest BIOS) and drivers. The X540 is in a PCI Gen3 x8 slot ( I set the slot to Gen2 in BiOS and no change). The version 28 (Feb 2022) drivers from Intel install and I am able to surf without issues.
The problem is that I can't get more that 1Gb speed on the card. It is connected to a Netgear 10Gb unmanaged switch at 10Gb with a CAT 6A 14ft cable. I have tested two different cables and no change.
Testing it to an iPerf3 local server (CentOS 7) with another X540 installed. I can't get past 2Gbs using the switch and also using a crossover cable (CAT 6A).
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 172.16.200.210, port 3839
[ 5] local 172.16.200.12 port 5201 connected to 172.16.200.210 port 3840
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 117 MBytes 984 Mbits/sec
[ 5] 1.00-2.00 sec 166 MBytes 1.39 Gbits/sec
[ 5] 2.00-3.00 sec 131 MBytes 1.10 Gbits/sec
[ 5] 3.00-4.00 sec 164 MBytes 1.37 Gbits/sec
[ 5] 4.00-5.00 sec 99.4 MBytes 834 Mbits/sec
[ 5] 5.00-6.00 sec 101 MBytes 846 Mbits/sec
[ 5] 6.00-7.00 sec 109 MBytes 917 Mbits/sec
[ 5] 7.00-8.00 sec 105 MBytes 879 Mbits/sec
[ 5] 8.00-9.00 sec 66.9 MBytes 561 Mbits/sec
[ 5] 9.00-10.00 sec 76.2 MBytes 639 Mbits/sec
[ 5] 10.00-10.04 sec 8.19 MBytes 1.57 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.04 sec 1.12 GBytes 955 Mbits/sec receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
My Internet is 2Gbe/2Gbe. The CentOS gets 2Gb/s up/down no issues. Windows under 1Gb/s.
Test from CentOS 7
Speedtest by Ookla
Server: Frontier - Los Angeles, CA (id: 14236)
ISP: Frontier Communications
Idle Latency: 3.11 ms (jitter: 0.77ms, low: 2.30ms, high: 3.44ms)
Download: 2337.37 Mbps (data used: 2.7 GB)
20.96 ms (jitter: 0.98ms, low: 3.58ms, high: 23.99ms)
Upload: 2545.85 Mbps (data used: 2.6 GB)
5.45 ms (jitter: 0.59ms, low: 3.10ms, high: 7.64ms)
Packet Loss: 0.0%
Also PROSET tool does not recognizes the card.
Intel README.txt says that driver has only be test under Windows 10 PRO
Supported Operating Systems
===========================
The drivers in this release have been tested with the following operating
systems (OSs). Additional OSs may function with our drivers but are not tested.
NOTE: Not all devices support all operating systems listed. Refer to the
release notes for detailed OS support information for your device.
Microsoft* Windows Server*, Azure Stack HCI, and Windows*
----------------------------------------------------------
Microsoft Windows Server 2022
Microsoft Windows Server 2019, Version 1903
Microsoft Windows Server 2016
Microsoft Windows Server 2012 R2
Microsoft Windows Server 2012
Microsoft Azure Stack HCI
Microsoft Windows 11 22H2
Microsoft Windows 11 21H2
Microsoft Windows 10 21H2
Microsoft Windows 10, Version 1809

Windows 10 was never designed for 10GE. Have you tried TCP Optimizer software https://www.speedguide.net/downloads.php ? It will tweak the receive windows and other TCP settings for high bandwidth.
The other thing you need to verify is that you have JUMBO frames enabled everywhere. For 10GE you should have a 9000byte MTU set and ensure the switch allows jumbo frames.
Also remember that the bottom two slots are x8 physical but only x4 electrical, based on page 1-3 of the manual https://www.supermicro.com/manuals/motherboard/C612/MNL-1585.pdf

 

[bill001g]

Since you are already running iperf it seems you are above your average user who posts here. To see if it is a hardware issue or more likely some garbage in windows i would try to boot a linux image on a usb stick. Most these images have support for the common 10gbit nics. Still this will only give you a clue. If it still doesn't work then you consider some hardware limitation. If it works then it is in the massive problem of finding the setting or driver buried deep in windows that is cause.

A note on iperf have you tried to swap the server and client so the traffic goes in the other direction. It "should" make no difference.

 

[silvexphoto]

Windows 10 was never designed for 10GE. Have you tried TCP Optimizer software https://www.speedguide.net/downloads.php ? It will tweak the receive windows and other TCP settings for high bandwidth.
The other thing you need to verify is that you have JUMBO frames enabled everywhere. For 10GE you should have a 9000byte MTU set and ensure the switch allows jumbo frames.
Also remember that the bottom two slots are x8 physical but only x4 electrical, based on page 1-3 of the manual https://www.supermicro.com/manuals/motherboard/C612/MNL-1585.pdf

I did tried on slot 7 Gen3 X8 and made no difference. The most I could get on slot 7 was 5Gb/s. I also get that speed once in while when doing iPerf test on slot 3 (current slot). If I do any speedtest (Ookla, google etc etc ) I never get past a gig.

 

[silvexphoto]

Since you are already running iperf it seems you are above your average user who posts here. To see if it is a hardware issue or more likely some garbage in windows i would try to boot a linux image on a usb stick. Most these images have support for the common 10gbit nics. Still this will only give you a clue. If it still doesn't work then you consider some hardware limitation. If it works then it is in the massive problem of finding the setting or driver buried deep in windows that is cause.

A note on iperf have you tried to swap the server and client so the traffic goes in the other direction. It "should" make no difference.

I have done that and no change. I did boot off a Linux USB stick and BOOM! 9Gbs! This is the reason I contacted Intel and they are digging into it.

 

[bill001g]

This is the stuff I hate the most. IPERF is a simple tool so it tends to not be affected by much other than the hardware and the driver. The only exception would be something like firewall or other software filtering/monitoring the data. You should see a cpu core spike if that was the case. Something like cfosspeed could also do that but this is more commonly unintentionally installed with the motherboard bloatware. I don't think supermicro bundles this garbage.

You could try older/newer drivers but that chipset is so old I don't know if they have been changing that much. Since it works under linux at least you know that it is not some bios setting....well you hope microsoft is not messing with bios settings.

 

[silvexphoto]

This is the stuff I hate the most. IPERF is a simple tool so it tends to not be affected by much other than the hardware and the driver. The only exception would be something like firewall or other software filtering/monitoring the data. You should see a cpu core spike if that was the case. Something like cfosspeed could also do that but this is more commonly unintentionally installed with the motherboard bloatware. I don't think supermicro bundles this garbage.

You could try older/newer drivers but that chipset is so old I don't know if they have been changing that much. Since it works under linux at least you know that it is not some bios setting....well you hope microsoft is not messing with bios settings.

I did try older drivers. From the first release to latest v28.x no change. Today I ran iPerf and was getting about 5Gb/s for about two runs. Then back to "normal" under 1Gb/s

 

[silvexphoto]

Running a local iperf server I get full 10Gb/s

c:\Users\esilva\bin>iperf3.exe -c localhost
Connecting to host localhost, port 5201
[ 4] local ::1 port 3098 connected to ::1 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 1.26 GBytes 10.8 Gbits/sec
[ 4] 1.00-2.00 sec 1.26 GBytes 10.8 Gbits/sec
[ 4] 2.00-3.00 sec 1.21 GBytes 10.4 Gbits/sec
[ 4] 3.00-4.00 sec 1.25 GBytes 10.7 Gbits/sec
[ 4] 4.00-5.00 sec 1.26 GBytes 10.9 Gbits/sec
[ 4] 5.00-6.00 sec 1.20 GBytes 10.3 Gbits/sec
[ 4] 6.00-7.00 sec 1.24 GBytes 10.6 Gbits/sec
[ 4] 7.00-8.00 sec 1.24 GBytes 10.7 Gbits/sec
[ 4] 8.00-9.00 sec 1.27 GBytes 10.9 Gbits/sec
[ 4] 9.00-10.00 sec 1.26 GBytes 10.8 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 12.4 GBytes 10.7 Gbits/sec sender
[ 4] 0.00-10.00 sec 12.4 GBytes 10.7 Gbits/sec receiver