Copying B-->A Kills Internet and network but A-->B does not

[Tanyac]

A=My PC (Windows 10 1703)
B=Server (Windows Server 2012 R2)

If I copy a large file from A to B there is little to no demonstrable impact on the network or the internet. If I copy a file from the B to A it kills the internet (Which is NBN FTTP 100Mbps down, 40 Mbps up).

All transfers, gaming and web surfing are dead while the copy is going. Ping to www.yahho.com go from 113ms to frequent timeouts, 2000 - 3000ms and I can't browse the network.

 

[Alabalcho]

My theory: Your LAN is 1GBit/sec, and copying off server takes it down (I can do that with my office network as well).

 

[Tanyac]

Found a work around, but I'd prefer to find the real cause and a real solution

I did extra tests with 4 PCs. With uTorrent running on PC A, and copy from B to A, C or D killed the network and the internet (I assume the issue would be the same with FTP and/or HTTP downloads too).
When my ISP blamed my router for Internet slowness I upgraded to an 8 port gigabit router, and removed a Netgear smart switch between the router and the patch panel. As I don't use uTorrent much, (mostly when I want to download a new Linux distro or some such), it went undetected.

Plugged my router into the Netgear GS108T-V2. Plugged all patch ports into the switch.

I can copy files from the server at several PCs at once with Utorrent downloading @ 88Mbps (11MB/s), play online games, stream a TV show or NBA League pass game, surf the web and my ping stays at a constant 113ms to www.yahoo.com.... That's how it's always been.

For me, this resolved the issues I was having, apparently as a workaround rather than a solution as my network appears to be one of a kind (Router --> Switch --> Device). Please do not try this on your networks. It apparently will not work.

 

[bill001g]

I suspect it is just a coincidence that it fixed it or you had some other issue.

If you where to open most routers up you would find a small switch chip running the lan ports. This in many cases was the same chip used by say a small 5 port external switch. What you have in effect is a switch with a single cable going to the router chip leaving 4 ports available to you for end devices.

The switch chip in general can run all ports at maximum speed up and down at the same time. So with 4 gigabit ports a total of 8gbit of traffic could pass at the same time....not that there is a realistic configuration that would use that.

Hard to say why you fixed it but it is not because you use a external switch.

 

[Tanyac]

I suspect it is just a coincidence that it fixed it or you had some other issue.
Hard to say why you fixed it but it is not because you use a external switch.

I can reproduce this issue. Take the switch out, plug in everything to the 8 port router, BAM.. Same problem.
I can run uTorrent from any PC (Windows 7, 10, 2012 R2), and copy from the server to any PC and the problem will surface.

The way it now is how I had it for years. It was only when I bought the 8 port router I figured why have an extra point of failure (switch). Never had a problem for over 10 years with a smart switch in place.

The advice was given by people on dedicated specialist networking forums.

So, in your expert opinion then, if it's not the switch that's intelligently routing traffic (It should be it's a smart managed switch), then what's causing the problem?

Realistic configuration? I have 10 PCs, 3TVs, 3 PS3/4, other media devices and 2 laptops all running wired. I do a lot of video editing and store the results on my server. When I want to work on the file, I copy it back to one of the editing PCs...

Perhaps then we will never know what the real cause is. But at least it's now the same as it was for the last 10+ years and everything is working perfectly now.

 

[bill001g]

Just because it happen to work for you and you can not explain why it works does not make a valid recommendation for other people. It is as bad as if I say I poke myself in the eye with a sharp stick and it makes my network run faster so you should also.

It SHOULD not make any difference and for the vast majority of people it does not cause any issues. So something in unique to your case but if not plugging it into the router fixes it for you then it likely is easier to just do that than to try to find the actual cause.

The only issue I have with your post is you recommending "never plug devices directly into the router". with no other basis that "it works for me" This is how we get so much misinformation on the internet.

 

[Alabalcho]

@Bill - I can reproduce it as well. Even that our router (Netgear running Tomato) has all gig ports, as long as high-volume traffic from server to desktop runs thru the router, the router "dies" for Internet access.

 

[Tanyac]

The only issue I have with your post is you recommending "never plug devices directly into the router". with no other basis that "it works for me" This is how we get so much misinformation on the internet.

That's extremely harsh! I do not have a malicious bone in my body and would never knowingly spread misinformation.

I can reproduce this with any of half a dozen programs running, on all 10 PCs on the network, with several switches. Given that it only started when I removed the switch, then "conceivably" the issue has existed for near on 20 years. So that means every version of Windows server since 2000 and all Windows OS from Windows 98

With that in mind, network design is not that hard. Internet Terminating device --> Router --> Switch --> device. All cabling Cat6 throughout.

So then, rather than just beating me over what worked for me, please help me solve what the real problem is.

 

[bill001g]

The only issue I have with your post is you recommending "never plug devices directly into the router". with no other basis that "it works for me" This is how we get so much misinformation on the internet.

That's extremely harsh! I do not have a malicious bone in my body and would never knowingly spread misinformation.

I can reproduce this with any of half a dozen programs running, on all 10 PCs on the network, with several switches. Given that it only started when I removed the switch, then "conceivably" the issue has existed for near on 20 years. So that means every version of Windows server since 2000 and all Windows OS from Windows 98

With that in mind, network design is not that hard. Internet Terminating device --> Router --> Switch --> device. All cabling Cat6 throughout.

So then, rather than just beating me over what worked for me, please help me solve what the real problem is.

Ok I guess I was not in a good mood sorry. I guess certain equipment could work that way but most routers you find have separate switch chips...even though lately it is on the same physical silicon it still functions as 2 devices.

My statement is based on the fact that lan traffic is mac addresses based and never leave the switch chip. It uses asic to be able to not delay the traffic.

What I can't see is why if you place the same chip in a external box it works but if it is in the router case it does not. All I can think of is that the traffic is somehow being processes through the router chip. Since the router chip is software based rather than asic it can not handle the data.

Router manufacture have actually figured out how to move more of the function to the switch chip, the NAT acceleration is one of the newer ones.

All I am saying is the router manufactures are designing the product to not delay traffic so it is very surprising that you have a unit that delays it.

 

[anotherdrew]

My statement is based on the fact that lan traffic is mac addresses based and never leave the switch chip. It uses asic to be able to not delay the traffic.

Interesting conversation. Forgive me if this naive, but most routers allow applying firewall rules to LAN traffic (as well as routing, etc) so doesn't that require layer 3 communication, where as a switch is all layer 2. Would that be enough to overwhelm the router's processor?

 

[bill001g]

My statement is based on the fact that lan traffic is mac addresses based and never leave the switch chip. It uses asic to be able to not delay the traffic.

Interesting conversation. Forgive me if this naive, but most routers allow applying firewall rules to LAN traffic (as well as routing, etc) so doesn't that require layer 3 communication, where as a switch is all layer 2. Would that be enough to overwhelm the router's processor?

Actual layer 3 switches can actually do some filtering. Anything that delays traffic like traffic shaping is not done but traffic policing can be done.

In most cases consumer router you can only filter traffic going lan-wan which passes through the CPU chip. Most devices use very simple layer 2 implementations that are only used to pass lan-lan traffic.

What I have found recently is you get a massive speed hit on the router lan-wan traffic if you even use simple firewall rules.

This is because the router manufactures have somehow figure out how to do NAT in the asic chips. Not sure how they accomplished that because the switch must now change IP and recalculate the checksums in the headers which takes some time.

When you turn the firewall rules on....or even simple feature like traffic utilization..it goes back to use the CPU chip to do the NAT.

Even a couple of years ago I would have said NAT had to be done by the CPU and it still appears that way on commercial switches I use.

I would suspect consumer router manufactures could put lan-lan filtering in the asic if they wanted but there is little demand......Many home user just care if they can surf the web and the router works.

 

[AndrewBC]

Well, whichever way you slice it, there is evidently some incompatibility between that particular switch and your system.

Have you tried borrowing a different brand (recent) switch and seeing if it still happens?

I had problems using one particular unmanaged switch as an intermediate switch between my router and some stuff on my homenet and had odd, intermittent problems (switch was newish and had worked fine before). Could not pin it down, all tests OK. Gave up & got a different switch and problems went away.

Mac

 

[Tanyac]

OK here are the facts straight from the horses mouth.

This IS a router issue. There is no nefarious problem lurking underneath that the addition of a managed switch "hides"

This router, an ASUS RT-AC88U is an 8 port router. It's the first 8 port router I've owned. I have always used 4 port routers and a switch which is why I've never come across the problem. This router is a "Store and Forward" router.

(1). Whilst not highly common, some 8 port routers to have 2 x 4 port switches. These switches are connected by a 1Gbps link. In my case, ports 5-8 are handled by a separate Realtek chip.

(2). The CPU and the switch are both connected by a 1 Gbps link.

Ports 1 - 4 on the router are on one switch chip and ports 5 - 8 are on the other switch chip.
When data traverses the two different switches (example, ports 1 and 5), that 1Gbps link will become the bottleneck

Additionally, as the CPU is bridged, and not the switch, the CPU prioritizes the local traffic over the remote traffic (Effectively cutting off all Internet traffic in favor of the local file copy). As you noted above, the CPU gets involved in the processing of the data, and since the file copy soaks up the 1Gbps CPU link, everything else stops.

2 bottle necks.

There are two solutions

(1). If there are 4 or less LAN segments put all segments on ports 1-4 or 5-8 so everything is on one internal switch. The CPU bottle neck does not occur in this situation and the switch can handle pretty much anything I threw at it.

(2). If there are more than 4 LAN segments you can shift the load from the router to an external switch. This handles the load, and as the router has only 1 active LAN port, there is no bottleneck.

However, if you have more than 4 LAN segments connected to this router, data will inevitably cross the two switches and things will grind to a halt. It seems the sensible thing to do in this situation is to offload the work to a separate switch, and let the router be just a router.

The switches that I am using are the Netgear Prosafe smart switches. Ideal for the environment I have.

Problem solved. I have wasted more than 3 days trying to track down some "other" problem that just doesn't exist.

You are entitled to believe whatever you want, but if you want verification try contacting ASUS and/or Netgear, and the ASUSWRT Merlin firmware forums.

 

[bill001g]

I appreciate your detailed explanation. You would think on a router that expensive they would use a switch chip that could handle considering you can get a 8 port external switch for $25.