Question Issues with upload speed with 2.5 Gbps connection ?

[winoni71]

I recently got a 2.5/1 Gbps FTTH line, but I immediately encountered a problem with the upload speed.

Below are all the details:

ONT: "Sercom FG1000R" with 2.5 Gbps LAN output
Router: "Asus GT-AX6000" with 1 WAN and 1 LAN 2.5 Gbps port (Drivers updated to the latest version)
Network card on "Asus ROG Crosshair VIII Hero (WI-FI)" motherboard: Realtek RTL8125-CG 2.5 Gbps (Drivers updated to the latest version)

The upload bandwidth is limited to approximately 300 Mbps only if the router is connected to the "Realtek RTL8125-CG 2.5 Gbps" LAN port of the PC, but exclusively if this has the negotiation setting to "2.5 Gbps Full Duplex".

If instead I set "1.0 Gbps Full Duplex", the upload speed is correctly saturated, the same happens if I connect the router to the PC from one of the 1 Gbps LAN port.

Evidently there is a communication problem between the router's WAN 2.5 and the PC's WAN 2.5 only in "2.5 Gbps Full Duplex".
I disabled all the options in the options in "Device Manager", but it didn't help.

With the ONT directly connected to the PC the problem appears but only with the speedtest you find below, with other tests however the upload speed is close to 900 Mbps.

Below are the speed tests in all the situations described above.

 

[winoni71]

ONT->PC LAN 2.5 (2.5 Gbps Full Duplex)

Idle Latency: 14.01 ms (jitter: 0.51ms, low: 13.53ms, high: 14.40ms)
Download: 1481.46 Mbps (data used: 2.5 GB)
42.79 ms (jitter: 49.68ms, low: 12.47ms, high: 511.01ms)
Upload: 169.53 Mbps (data used: 229.4 MB)
13.68 ms (jitter: 3.52ms, low: 11.94ms, high: 253.28ms)
Packet Loss: 0.4%

ONT->PC LAN 2.5 (1.0 Gbps Full Duplex)

Idle Latency: 13.64 ms (jitter: 0.28ms, low: 13.48ms, high: 14.01ms)
Download: 936.53 Mbps (data used: 1.1 GB)
59.60 ms (jitter: 55.28ms, low: 12.60ms, high: 528.78ms)
Upload: 204.55 Mbps (data used: 353.1 MB)
13.44 ms (jitter: 2.76ms, low: 11.64ms, high: 233.16ms)
Packet Loss: 1.3%


Router LAN 2.5->PC LAN 2.5 (2.5 Gbps Full Duplex)


Idle Latency: 13.02 ms (jitter: 0.10ms, low: 12.69ms, high: 13.10ms)
Download: 1463.82 Mbps (data used: 2.2 GB)
48.51 ms (jitter: 53.90ms, low: 11.31ms, high: 264.39ms)
Upload: 217.48 Mbps (data used: 325.3 MB)
13.27 ms (jitter: 18.91ms, low: 11.37ms, high: 246.96ms)
Packet Loss: 0.5%


Router LAN 2.5->PC LAN 2.5 (1.0 Gbps Full Duplex)


Idle Latency: 13.01 ms (jitter: 0.10ms, low: 12.96ms, high: 13.24ms)
Download: 939.94 Mbps (data used: 1.2 GB)
100.95 ms (jitter: 63.68ms, low: 13.94ms, high: 578.90ms)
Upload: 698.64 Mbps (data used: 1.1 GB)
13.44 ms (jitter: 1.28ms, low: 11.37ms, high: 36.14ms)
Packet Loss: 0.0%

Router LAN 1.0->PC LAN 2.5 (1.0 Gbps Full Duplex)


Idle Latency: 13.02 ms (jitter: 0.10ms, low: 12.93ms, high: 13.14ms)
Download: 946.44 Mbps (data used: 1.2 GB)
81.70 ms (jitter: 58.20ms, low: 12.07ms, high: 526.92ms)
Upload: 782.08 Mbps (data used: 1.3 GB)
17.85 ms (jitter: 8.81ms, low: 11.44ms, high: 233.39ms)
Packet Loss: 1.2%


Router LAN 2.5->PC LAN 1.0 (1.0 Gbps Full Duplex)

Idle Latency: 13.00 ms (jitter: 0.14ms, low: 12.85ms, high: 13.17ms)
Download: 872.45 Mbps (data used: 1.6 GB)
107.77 ms (jitter: 71.12ms, low: 11.56ms, high: 1217.69ms)
Upload: 660.39 Mbps (data used: 1.0 GB)
12.61 ms (jitter: 1.26ms, low: 11.37ms, high: 41.98ms)
Packet Loss: 23.1%

Router LAN 1.0->PC LAN 1.0 (1.0 Gbps Full Duplex)

Upload: 656.68 Mbps (data used: 1.1 GB)
13.21 ms (jitter: 1.76ms, low: 11.43ms, high: 90.97ms)
Packet Loss: 18.7%

 

[USAFRet]

Ah, so the posting now works?

 

[winoni71]

Ah, so the posting now works?

Yes, something in the last lines, I have no idea what.
Maybe it was too long.

 

[winoni71]

To find out which is the guilty party, the network card (my suspect) or the router, I want to try a different card on my PC.
What do you recommend?
From searching it seems that Intel is the best brand, but I'm not sure.

Thank you.

 

[bill001g]

Now that intel has gotten their act together their 2.5g chipset seems to work. If you were a very early adopter there were a bunch of motherboards that had horrible performance and it couldn't really be fixed with firmware.

Luckily the price of 2.5g nics are fairly cheap.

Before you try this get a linux USB boot image. This will quickly show you if it is a strange windows issue or a driver issue.

 

[winoni71]

Before you try this get a linux USB boot image. This will quickly show you if it is a strange windows issue or a driver issue.

Just tried it, same results.
Which Intel chipset do you recommend?
I was thinking of getting the "QNAP QXG-2G1T-I225".

 

[bill001g]

You will pay a premium for the qnap name on the board. i225-v2 is the one I think that fixed their issues. They stopped manufacting the older v1 so it is unlikely you find them. They also have a i226 that came out about a year ago. More reports of issues but I think they fixed the 226 with firmware. It is strange that a huge company like intel that has made nic cards for many years screws things up so bad.

 

[winoni71]

It was the God damn router all along...
Any suggestions on a substitute?

The "ASUS ROG Rapture GT-AX11000 PRO" seems good, even though Asus shouldn't get a cent from me after this debacle.

What is the point of a 10 Gbps LAN port when the WAN port is limited to 2.5 Gbps?

 

[bill001g]

Very hard to find routers that have multiple 2.5g or faster ports. You could look at a number of the small pc devices that have multiple 2.5g ports. There are many linux router/firewall images.

In some ways a small pc is a better choice for these faster connections. You can't actually use any of the routers other features like say parental controls or data filtering . Most modern routers use special asic based NAT that allows the traffic to bypass the cpu chip. If the traffic needs to be examined in anyway the cpu must now do that function as well as the NAT function.

The CPU in routers is very small so they quickly limit your traffic rates.

If you have not tried it already factory reset your current router. After that set only the admin and wifi passwords. This would eliminate any strange setting you might have made that causes the router cpu to get involved in the traffic.
There are many routers that have say 2.5 or faster wan port but only 1gbit lan ports. The idea is multiple machines will together be able to use the total 2.5g bandwidth. Hard to say very few people actually need even 1gbit internet connections which is why routers with multiple high speed ports are hard to find.

 

[winoni71]

If you have not tried it already factory reset your current router...

Tried it, anyway I have already shipped it back.
It's an issue that this router has with upload speeds, I found many posts in various sites of people complaining about it.

 

[winoni71]

Ok, I am out of ideas...
I have now tried 2 different routers and 3 different network cards (2x 2.5Gbps 1x 10Gbps).
Consistently I get low upload speeds with anything other than a "1.0 Gbps Full Duplex" negotiation setting, the download speed on the other hand is correctly improved by higher negotiations.

What am I missing?
Everything in the system, past and present, is clearly capable of going over 1Gbps in download and at 1Gbps in upload, so what gives?
What could possibly be the problem?
Could it be the ONT, some BIOS setting, some OS setting?

At this point I don't care anymore to achieve the goal, I'm just curious to find out what the problem is.

Thank you.

 

[bill001g]

Do you have a second machine that run over 1gbit. You could run test data between the machines directly, using a switch, and then even though the router. You can if you work at it hook a machine to the wan port and then pretend it is on the internet. That way you can force test just the router without all the extra stuff involved with the internet itself.

In general you want to leave the setting on the nic in auto mode. I am not sure at 2.5g speeds but I know at other speeds if one end is set to auto and the other is set to some fixed value you get can get inconsistent results. The ports are using electrical signalling to get the speed and when they run auto they expect to get these signals. This though was mostly a issue back in the days of 100mbps ports being full or half duplex. Gigabit and faster do not have the concept of half duplex to worry about.

 

[winoni71]

Do you have a second machine that run over 1gbit

I don't.

In general you want to leave the setting on the nic in auto mode.

I tried everything, from leaving it to auto to change it to manual, to disabling all sorts of extra settings, it made no difference.

 

[bill001g]

You would think this was the pc causing this but you have changed so much. If the routers just didn't work you would see massive numbers of posts about it.

 

[winoni71]

What else could I try?

 

[bill001g]

You have already done almost everything I can think of. You have used a different OS which eliminates most software. You have used a different nic and router which eliminates almost all the hardware.

The only thing I guess that stays the same if I have not missed something is speedtest app. What does not make a lot of sense is why it would sometime work much better.

Note I get 930mbps up and down very consistently on 1gbit connection.

 

[winoni71]

You have already done almost everything I can think of. You have used a different OS which eliminates most software. You have used a different nic and router which eliminates almost all the hardware.

And now I managed to test it with another PC, using the 10 Gbps card and the results are exactly the same.

The only thing I guess that stays the same if I have not missed something is speedtest app.

I tried different apps and sites, the only common denominator, although I can't see why that would be the issue, is the ONT.

What does not make a lot of sense is why it would sometime work much better.

Note I get 930mbps up and down very consistently on 1gbit connection.

I too do get the same speeds, but only if the negotiation is set to "1.0 Gbps Full Duplex".
With 2.5 or 10 Gbps I get higher DL speeds but the UL speed is capped at around 300 Mbps.

I'll contact my ISP regarding the ONT, nothing else I can try.

 

[bill001g]

I assume since you are hooking the 10g card to the router it will also run at 2.5g.....not all 10g devices can.

With 2 machines you can test your equipment better. I would use a old line mode tool called IPERF to test since you can't use speedtest. You could copy files but at very high speeds you could be affected by disk subsystem issues.

I would first hook them back to back, most times the port will come up using auto mode. You do not use cross over cables even though you still see that wrong information. You will manually need to set the IP addresses.

Next you could try hooking on both sides of your router. You would use 1 machine too be wan and the other lan. You will again have to set ip addresses and in this case also the wan address on the router. You want to use different subnets.

This will see if you can transfer the data via the router.

I would immediately have blamed your ONT but when you plug a router in between the connection between the ONT and the router is always the same. It will not know that you have changed the lan to run at 1gbit rather than 2.5g. It would be very strange if the ONT was the cause. Mostly I would test so that the ISP does not come all the way out and then say everything is fine when they plug in their test machine.

 

[winoni71]

I assume since you are hooking the 10g card to the router it will also run at 2.5g.....not all 10g devices can.

The one I now own has a 10G and a 2.5G both LAN/WAN, so in my tests I connected the router to the 10G card using the 10G port of the router and the negotiation was correctly at 10 Gbps.

I will try what you suggested, although I'm starting to lose interest and patience at this point.

 

[winoni71]

I would use a old line mode tool called IPERF

Here are the results:

2.5Gbps card->Router->10Gbs card

[ ID] Interval Transfer Bandwidth
[332] 0.0- 1.0 sec 284 MBytes 2380 Mbits/sec
[332] 1.0- 2.0 sec 284 MBytes 2380 Mbits/sec
[332] 2.0- 3.0 sec 284 MBytes 2380 Mbits/sec
[332] 3.0- 4.0 sec 282 MBytes 2363 Mbits/sec
[332] 4.0- 5.0 sec 283 MBytes 2375 Mbits/sec
[332] 5.0- 6.0 sec 284 MBytes 2379 Mbits/sec
[332] 6.0- 7.0 sec 282 MBytes 2366 Mbits/sec
[332] 7.0- 8.0 sec 284 MBytes 2379 Mbits/sec
[332] 8.0- 9.0 sec 282 MBytes 2366 Mbits/sec
[332] 9.0-10.0 sec 278 MBytes 2329 Mbits/sec
[332] 0.0-10.0 sec 2825 MBytes 2370 Mbits/sec

10Gbs card->Router->PC 2.5Gbps card

[ ID] Interval Transfer Bandwidth
[392] 0.0- 1.0 sec 166 MBytes 1392 Mbits/sec
[392] 1.0- 2.0 sec 254 MBytes 2134 Mbits/sec
[392] 2.0- 3.0 sec 254 MBytes 2128 Mbits/sec
[392] 3.0- 4.0 sec 277 MBytes 2320 Mbits/sec
[392] 4.0- 5.0 sec 283 MBytes 2376 Mbits/sec
[392] 5.0- 6.0 sec 181 MBytes 1522 Mbits/sec
[392] 6.0- 7.0 sec 274 MBytes 2295 Mbits/sec
[392] 7.0- 8.0 sec 268 MBytes 2250 Mbits/sec
[392] 8.0- 9.0 sec 260 MBytes 2177 Mbits/sec
[392] 9.0-10.0 sec 263 MBytes 2209 Mbits/sec
[392] 0.0-10.0 sec 2486 MBytes 2080 Mbits/sec

The "2.5Gbps card->Router->10Gbs card" is more stable in speed than the other way around, I ran it for 120 seconds and it occasionally dropped below 2000 Mbits/sec, anyway both would be fine for my needs.

So now we have established that the speeds can be achieved, and so we are back to square one.

 

[bill001g]

I guess though you know that all your equipment is good. This leave something strange with the ONT or something about the testing.
There used to be a site you could run IPERF from on the internet. I don't know what the maximum rate it supports.

 

[nigelivey]

"What is the point of a 10 Gbps LAN port when the WAN port is limited to 2.5 Gbps?"

Not everything transverses from LAN to WAN.....

 

[bill001g]

"What is the point of a 10 Gbps LAN port when the WAN port is limited to 2.5 Gbps?"

Not everything transverses from LAN to WAN.....

This is just his latest testing. His original issue is he is using a 2.5g machine and hooking it to a 2.5g internet connection and getting far less than that on upload but it is not even consistently less. His last test pretty much show it is something with his fiber connection or the ONT box.

 

[gggplaya]

Could still be the router, which relies on hardware NAT to achieve those speeds. If it has to use software NAT for any of the ports, then 300-500mbps is about right depending on which ARM processor it's using.

That's why I built my own x86 router. It's software NAT and software QOS, but it can move traffic well beyond 2.5gbps.