Question Server ethernet ports dropping connection randomly ?

[Viking2121]

So I had a lightning strike my house and took out many devices, my server was one of those said devices, Anyway I got me a new used server from work, older but it works, Its got an Intel S2400SC motherboard with dual E5-2470s, 128GB of ECC ram, and a 500watt Seasonic gold PSU.

Anyway, I have it hooked up with both ethernet ports threw a 5 port D-Link cheapo dumb switch, I have, I have TrueNAS installed on it for file shares, TrueNAS I have it set up to use the 2nd Ethernet port, and Plex with all my media files uses the 1st ethernet port, that way I can move big files without causing problems with Plex.

My problem is, the ports will just randomly drop, at different times, Like the 1st nic will go down for a few seconds and then maybe the other nic will, I don't get it, I switch my switch to an 8-port TP-Link managed switch and that did not help and I switched back to the D-Link because I use the lights in front of it for mostly troubleshooting.

I got my main rig also hooked up to the switch and it has zero issues at all, I guess the only thing I did;t change was the ethernet cables, but I can't see both being faulty just randomly.

I took the TrueNAS drive out and installed Ubuntu on a spare drive and ran it for a while, and even setup plex on it so I can get a consistent load don't the network, 1 user or 5 users it will still just out of the blue drop the ethernet port altogether, Ubuntu acts like I just unplugged it.

I tried an SMC 10/100 USB nic, and while slow I didn't experience any just random drops.

Any Ideas? Its not a big deal to buy a dual or even faster nic as an add-on, I don't see why the onboard nics are dropping out like that, The Tech at work has no idea if it had issues, could be the reason why it was retired he said, makes sense.

Thanks

 

[bill001g]

Do do you have 1 network or do you have 2 separate networks ie subnets.

To some extent it depends on how the server works and if you actually have 2 different virtual machines.

If you were to just hook up a computer running say window or linux with 2 nic cards to the same network you can get a loop condition that would crash your whole network. Many devices attempt to prevent this condition by block all but 1 port. One of the common methods is called spanning tree.

So if you are attempting to running virtual machines it could be some issue with how that is implemented. If the main OS does not have a way to keep these separate you can have problems. If you are attempting to use any form of port bonding then the switch as well as the server/computer must support that. A unmanaged switch does not support this and your manged one might. The most common form is 802.3ad but it may or may not work well since it does not balance the data based on load but on a math formula. It was designed for a central server where you have 100s of client machines accessing it not a few very large data transfers.

 

[Viking2121]

Do do you have 1 network or do you have 2 separate networks ie subnets.

To some extent it depends on how the server works and if you actually have 2 different virtual machines.

If you were to just hook up a computer running say window or linux with 2 nic cards to the same network you can get a loop condition that would crash your whole network. Many devices attempt to prevent this condition by block all but 1 port. One of the common methods is called spanning tree.

So if you are attempting to running virtual machines it could be some issue with how that is implemented. If the main OS does not have a way to keep these separate you can have problems. If you are attempting to use any form of port bonding then the switch as well as the server/computer must support that. A unmanaged switch does not support this and your manged one might. The most common form is 802.3ad but it may or may not work well since it does not balance the data based on load but on a math formula. It was designed for a central server where you have 100s of client machines accessing it not a few very large data transfers.

Both nics are on the same subnet, but dhcp is disabled, I set a static for both, 10.0.0.200 and the other 10.0.0.201, I normally wouldn't care to have 2 nics going, but I figured plex on 1 nic while moving files on the other would lessen the risk of clients buffering, especially once I get more 4k movies on it. But even 1 nic, it just drops as if I unplugged it, it's strange.

My old server that died I had 2 nics and was set up essentially the same way, same TrueNAS OS and worked great for almost 5 years. I'm just not sure why I'm having issues now with this server.

I think I'm just going to start over, reset everything back to default, router, server, and just try it again.

 

[bill001g]

You can have 2 nics on the same network but not on the same machine. This is where it gets messy if you are running virtual machines since they are only kinda on the same machine.

Unless you run lots of custom software what it does it will pick one of the 2 nics and send all traffic with that ip/mac address it will not use the second nic. I am somewhat surprised it let you set the IPs like that most times you get error messages.
Your switch might just be smart enough to avoid issues. If you want to test plug a ethernet cable between 2 ports on the switch. Cheaper units your will crash you whole network doing this because you get a broadcast traffic loop. If effect that is what is happening if you put both nics in the same subnet/network. The PC tends to also be smart enough to not allow this loop condition by not actually using both port.

Not sure why a single nic would go down. This is pretty simplistic stuff that is not really software related. It is either going to be a bad cable or a bad port. Bad cables are almost always the cause. There is a massive amount of fake cable being sold but even quality cables can be damaged or go bad. You only need cat5e but it must be pure copper (no cca) and have wire size 22-24 (no flat or thin cables).

 

[Viking2121]

You can have 2 nics on the same network but not on the same machine. This is where it gets messy if you are running virtual machines since they are only kinda on the same machine.

What do you mean, You can vary well have 2 NICs on the same PC, in fact, is quite common for server for redundancy purposes, Of course, teaming or a vlan or lag or whatever the term is to combine the nics is sorta a pain and but can be done if other device supported it.

They can also be in the same subnet, but in my case as a make-shift load balancer, I do not need teaming, so as long as they have a different local IP address it will work, If you skimped over the part where I had it like this for 5 or so years on a different server bored with 2 NICs with TrueNAS, it worked flawlessly, and Plex was using 1 nic, and file shares were using the other, or well I could access the file shares on either local IP actually but you get what I mean.

In fact, I did fix this problem btw, I went out and bought a new router as my old one once it was reset would not come back online and could not access its web interface at all on any device. My server which is running TrueNAS, with a Ubuntu VM for pihole, a windows VM for hosting a game that doesn't work on ubuntu, and Plex in a Jail, It all works fine.

TrueNAS can be accessed (Files shares as well) with 10.0.0.200 on nic #1 or 10.0.0.201 nic #2, Pihole can be accessed with 10.0.0.202 on nic #1, and the Windows VM can be accessed with 10.0.0.204 nic #1 with Remote Desktop or VNC (My other mini PC runs another Ubuntu server and Pihole at 203), Plex I have it on 10.0.0.205 on the same nic that's using 10.0.0.201 nic#2, All VMs have a different static IP's and all works fine and flawless just like my old server with just 2 1gbit nics, so yes 2 nics on the same PC is a thing and works perfectly unless Im completely missing something or don't understand something you said.

 

[bill001g]

You are being too literal here. I mistyped the second time when I said pc. I said you can't have 2 nics on the same machine which is different than 2 machine on the same pc. When you start talking about running "virtual" machines it gets even more muddled what words mean.

There also is special software in some devices that can tie a application to a certain nic card which is very similar to how virtual machines are done.

So with the default configuration on linux and especially on a windows install it will only use 1 of the nic cards. Why it doesn't function in other cases is a problem with whatever custom or simulation software you are running to get past the multiple nic issue.

 

[ventanni]

So I had a lightning strike my house and took out many devices, my server was one of those said devices, Anyway I got me a new used server from work, older but it works, Its got an Intel S2400SC motherboard with dual E5-2470s, 128GB of ECC ram, and a 500watt Seasonic gold PSU.

Anyway, I have it hooked up with both ethernet ports threw a 5 port D-Link cheapo dumb switch, I have, I have TrueNAS installed on it for file shares, TrueNAS I have it set up to use the 2nd Ethernet port, and Plex with all my media files uses the 1st ethernet port, that way I can move big files without causing problems with Plex.

My problem is, the ports will just randomly drop, at different times, Like the 1st nic will go down for a few seconds and then maybe the other nic will, I don't get it, I switch my switch to an 8-port TP-Link managed switch and that did not help and I switched back to the D-Link because I use the lights in front of it for mostly troubleshooting.

I got my main rig also hooked up to the switch and it has zero issues at all, I guess the only thing I did;t change was the ethernet cables, but I can't see both being faulty just randomly.

I took the TrueNAS drive out and installed Ubuntu on a spare drive and ran it for a while, and even setup plex on it so I can get a consistent load don't the network, 1 user or 5 users it will still just out of the blue drop the ethernet port altogether, Ubuntu acts like I just unplugged it.

I tried an SMC 10/100 USB nic, and while slow I didn't experience any just random drops.

Any Ideas? Its not a big deal to buy a dual or even faster nic as an add-on, I don't see why the onboard nics are dropping out like that, The Tech at work has no idea if it had issues, could be the reason why it was retired he said, makes sense.

Thanks

I'm not an IT guy, but I am a low voltage cable technician.

Assuming you're not creating packet storm issues as the above posters mentioned, the problem is likely being caused by bad cabling, specifically Near-End Crosstalk (aka NEXT) issues. NEXT occurs because there are four pairs of copper wires inside your network cables, and there is always some electrical signal that crosses from one pair to the next. What mainly separates CAT5 from CAT5e and CAT6 and beyond is how well it minimizes NEXT. NIC's will respond to NEXT issues by attempting to re-establish signaling, but when the NEXT is too bad, you simply won't get a connection whatsoever. I'm going to make an educated guess that you're probably using the same patch cables, and this is common with cables that are affected by lightning strikes. Even if you aren't, this is a common issue with patch cables in general.

The reason you're not seeing it with the 10/100 NIC is because faster network speeds like 1gb and 10gb require much tighter NEXT tolerances.