Gigabit Ethernet: Dude, Where's My Bandwidth?

[iwod]

Thanks, I have been calling for an articel to test Theoretical bandwidth using Ram Drive to Ram Drive over the network. And yet no site has ever published this.
So the questioned is finally answered, CAT 6 or CAT 5, Jumbo Frame Size, Network Equipment etc doesn't really matter. The biggest hurdle is our HDD.

 

[Guest]

Why have you not tried Solid State Disks?

 

[rand_79]

I didnt read all 6 pages of comments..

but if they had opened winamp or media player(among other programs) while transferring files microsoft's over aggressive network throttling would have limited you to around 10MB/sec anyway.

 

[TheKurrgan]

First to the guy speaking of Jumbo frames, fragmentation, TIA/EIA, this was about a HOME network, not a corporate network. That being said, from working in the industry heres how the Cat5/Cat6 breaks down:
Cat5e will not certify using ANSI standards for cable testing at 1Gbit when used for the horizontal cabling. With that out there, in practical tests it runs 1gb just fine, even at long cable lengths. Where it breaks down is in long runs with poor equipment at either end. In those scenarios it will show a difference in speed that is measurable. As to interference, electricity is the least likely candidate to cause any link degradation due to inducted interference. The frequencies the two operate at are far too different. However other higher frequency lines may cause interference, e.g in building BDA's for Cellular phones and back hauls for radio communications. This is where there will be a benefit for Cat6. Also when it comes to a home gbit network most switches are a 1 gb dumb switch, and probably wont even come close with every optimization you can find to sustaining more than 1 transfer at 980ish Mbit/sec
As to using a cat5e/cat6 patch cable, unless you've got some weird useless nic or have a huge amount of UHF and higher transmitters hanging around, you can use silver satin for a patch cable and attain the same speed (I've done it just to see)
Corporate networks however have much more to gain from GB networks. Jumbo frames combined with other optimizations along with multi-pathing make iSCSI a viable option for use as a SAN provided you're switches arent sold at Wal Mart.
As to other network considerations such as Jitter and the like which may only be noticeable by streaming sources such as Video over IP Surveillance systems, 100Mbit is preferred for the cameras / encoders as it survives with less problems with extremely sensitive applications.
I would like to see more comparisons between soho switches and higher end ones. For example, in my poor experimentation, I cut in half a spool of 1000ft cat5e, (this wire was used later for REAL purposes) and put ice cubes on each end, then plugged both in to an HP 5308 using a 16port 10/100/1000 module. Both machines where fairly high end HP workstations (xw8400 using onboard LSI raid in R0 config with 3 Velociraptors (replaced the stock seagate junker that came with them)
Throughput for this was right at 800Mbit/sec from either machine. They both ran Server 2003 R2. I then used the same machines and plugged them into a single Netgear off the shelf switch, and got only 270 at best. I then used regular patch cables on the netgear and poof, 730-800 was attained. File was 8.2GB insize being transferred.
If you guys ever get around to benching some Cisco vs HP stuff (at LEAST stackable switches, preferably layer 3) it'd rock
Copper 1GB Trunks VS Fiber for example wouldbe another cool one (2 copper = 1 fiber 1G trunk as I remember, so 4 copper members VS 2 1G Fiber to be fair)
Site rocks.
Thanks for the damn drop down list btw, sucked with out it on the site.

 

[tontito]

Probably raid5 would perform better because you would have at least 3 disks. Then the more disks you have, more speed you can get.

I did this test with a storage array with 10 disks i think and i could get almost maximum speed(was similar to disk memory).

 

[rooket]

cool now this means I don't have to run new wire in the house for gigabit. I'm only running 10/100 now because most wireless routers are cheap junk with only 10/100 built in.

 

[rkaye]

I wish you would have tested synchronous operation and its result on file transfer speed: sending a file (ram drive to ram drive) at the same time as receiving a file (ram drive to ram drive).

 

[rkaye]

I wish you would have tested synchronous operation and its result on file transfer speed: sending a file (ram drive to ram drive) at the same time as receiving a file (ram drive to ram drive).

 

[evilsizer]

I've never set up a network, so I assume this article is more for me than someone who's a Network Admin. The article made sense to my untrained mind and I actually learned a few things from it. The follow up posts were also informative since I have a grasp on tech lingo for the most part.

Flaming Mr. Woligroski only shows a lack in people skills. If you feel the article is wrong then point out the problems and explain your position in the same general feel of the article. That way the article grows with your knowledge added to it rather then having a bunch of tech egos spitting at each other.

We non-techies need to learn from somewhere. Tom's has always been the place for me. If you need a teched out article then write one. Please don't flame away the ones written more towards a base audience than a tech savvy one.

 

[triadwarfare]

Don't try vista. Use XP instead. I've noticed when I'm transferring a file from XP to Vista (Ex: transferring Left 4 Dead from my sister's nettop to my cousin's laptop which runs Vista Home Premium) It's a lot slower than transferring the same file from XP to XP (nettop to laptop).

Vista is a performance hag. It can even make file transfers slow "bullet-time" like the Matrix/Max Payne.

@Evelsizer
Have you ever used the internet? (Stupid question, of course you have!!! How stupid of me to ask cause you were able to post this message!) It applies to every average Joe who uses the internet. What isp do you use? Cable? DSL? FTTH? Dial-up?

Don't tell us that you never set up a network (unless you paid someone else to set your network up for you). Good luck with you!

 

[Metaspherz]

Thanks, Tom's Hardware for addressing my primary concern: should I replace my CAT5e cables with CAT6 cables to effectively increase my download and upload speeds? Based upon what I read: NO. I can effectively keep my current home network of 4 (one laptop without gigabit) PC's tethered to the CAT5e cables and stop worrying that I'm at a disadvantage. Everyone's in the same boat, as it were. All I really need are faster HDDs or a RAID array. That $100 laser gaming mouse doesn't seem to matter as much after all....

I'm by no means a "newbie" when it comes to home networking. I've hosted a dozen LAN parties over the years. But I feel like one when it comes to understanding the tech details and jargon that the professionals use to make themselves feel superior to the rest of us. I've been building gaming PC's for 25 years. Never bought one off-the-shelf. Retail gaming rigs cost double what I pay to build. I'm more interested in lag times too (on-line or LAN) because when it comes to gaming THAT's the difference between life and death: success or failure as a competitor. Speed is primary. It's the reaction time and NOT who's got the biggest gun. For instance, when you're in a QUAKE (just as an example) LAN party and you see an adversary you press that trigger button as fast as you can. AND IF NOTHING HAPPENS for a few micro seconds (lag time), that means that the other guy can get off a few rounds at you in the mean time and you're dust. THAT's VITALLY IMPORTANT to we competitors. So, although many here hype their superior knowlege of the whys and wherefors of networking, I'm happy that TOM's Kept It Simple Stupid....it's my harddrive that's the bottleneck (lag time) in my system and NOT my CAT5e cables--nor my antiquated itchy trigger finger. As always, it seems that the bottleneck has always been and always will be those pesky hard disk drives ....

 

[mariushm]

what a bad article... seriously...
The maximum throughput possible is 1gbps or 1.000.000.000 bits or 119 MB/s, not "about" 125 MB/s.

There's absolutely no mention of TCP/IP packet sizes and how many bytes are used by the TCP/IP in headers and extra information, so in reality you're never be able to get the maximum 119MB/s with TCP/IP, more like 111MB in the chart.

If you really want to do proper tests, every sane person reading the article would expect you defragment the drive before doing tests.

Also, you'd want to use an application that can read and write properly to drives. In Total Commander for example, you can configure it to read and write in 64KB, 128KB or 4MB chunks, which improves reading and writing compared to the default 32KB reads and writes Windows Explorer uses. A bit unrelated, but this is also the reason I was able to have 40MB/s+ reads and writes in the days of Barton processors through gigabit network when using DC++.
Nowadays, any HDD will be able to write on average with 60MB/s if you use the right application to transfer the files.

 

[Metaspherz]

**...this article is not for Tomshardware it is not meant for people that understand networking or maybe even computers. Pass this article on to another site with more "normal" visitors.** --MartenKL


Wrong, MartenKL, you are a nitwit! "Normal" persons? Care to define normal? Your pudding brained comment certainly suggests that you are a "normal" computer snob/elitist. You think that Tom's is just for nerd types, like yourself, whom pretend to know everything there is to know about computers. Therefore, normal means those non tech-savvy persons who just need simple info and don't belong here.

Is there a site dedicated to newbies as well as teckies? ...Oh, wait! It's called tomshardware dot com. Whoa! Noooo! Open your pea-sized brain and let some fresh air in numbskull. People come to Tom's because it's the best site on the web for all kinds of skill levels. I've been visiting Tom's for over 10 years. It's been a wealth of info for me and my peers. I've been working with computers for over 37 years. I've yet to hit the ceiling when it comes to knowlege about them or met someone who couldn't stand to learn more about them. Tech changes. That's why Tom's is here...for everyone!

Also, I think that most of the negative comments by certain techs (or wannabes) missed the point of the article entirely: Where's The/My bandwidth? The article adaquately answers that question without complicating it with a lot of unnecessary technobabble and minutia of detail. I don't care if the test hdd was defragged. I don't care about packet size or the quantity of files. In the real world it all varies and averages out in the long run. No test is absolutely complete and concise.

'Keep it simple stupid ' applies here. For that I'm grateful! All I wanted to know is which cable (cat5e vs cat6) suited my purpose best and if I could save money. I didn't require the whole history of computer science.

Thank you Mr. Woligroski for your simple and to the point perspective. Tom's is the best place for this article. Because this is where the smart 'normal' ones look first....

 

[prochobo]

Good test, but you guys left out one critical test:local disk write speed vs write speed over the network. When I did my own tests, I found that local disk to disk write speed was about 85MB/s, but network disk to disk speed was 41MB/s.

I was using an Intel Atom 330 platform, so it could have been that the system was not fast enough. That's when I looked to these tests since you guys have faster systems. However, you guys only did network file transfers and didn't compare them to local file transfers

 

[KyleSTL]

Good test, but you guys left out one critical test:local disk write speed vs write speed over the network. When I did my own tests, I found that local disk to disk write speed was about 85MB/s, but network disk to disk speed was 41MB/s.

I was using an Intel Atom 330 platform, so it could have been that the system was not fast enough. That's when I looked to these tests since you guys have faster systems. However, you guys only did network file transfers and didn't compare them to local file transfers

41MB/s sounds about right for any storage in an Atom-based system (either SSD or 4200/5400RPM 1.8/2.5" HDD). Remember the overall transfer rate will be determined by the slowest component in the system.

 

[crlx]

[citation][nom]Anonymous[/nom]First of all, gigabit ethernet uses entirely different addressing and encoding from 100-meg, and overhead is one heck of a lot greater than that.First of all, there's the 10b/8b encoding, so an 8-bit byte is encoded to a 10-bit unit. Then there's a concept of invariable frame sizes, whereit might be possible that a TCP/IP packet spans two frames, filling 100% of the first and 1% of the second, it means 50.5% efficiency. Third, every frame is payload only in part, rest is taken up by header information, footer and CRC. It's not much, perhaps about 5% of the frame, but it can get noticeable.First, you have to divide by 10, not by 8, to get the speed in bytes/second (ie. 100 MB/s, not 125 MB/s).Second, if you transmit a lot of inefficient frames (networking programs aren't exactly frugal about bandwidth when they have gigabit ethernet, and next to none are actually optimized in any way for it), you might lose up to half of the bandwidth.Third, when you factor in the frame level overhead, you might end up with maybe 40-45 MB/s of the promised 100 MB/s...Fortunately, a lot of these issues can be resolved by optimizing software and firmware to take advantage of the available bandwidth and idiosyncracies of gigabit ethernet.[/citation]
I say the "10b/8b encoding" is one of the easiest (to explain) but biggest overhead issue...!
(The impact compare to the easy calculation)
Though packet overhead is worth mention too; with small packet ~100bits the overhead comes close to give you just 70%-75 of bandwith _without_ counting the 10b/8b encoding schema, which just remove even more...

Sorry but this was a bad test..

 

[crlx]

[citation][nom]TheKurrgan[/nom]First to the guy speaking of Jumbo frames, fragmentation, TIA/EIA, this was about a HOME network, not a corporate network. That being said, from working in the industry heres how the Cat5/Cat6 breaks down:Cat5e will not certify using ANSI standards for cable testing at 1Gbit when used for the horizontal cabling. With that out there, in practical tests it runs 1gb just fine, even at long cable lengths. Where it breaks down is in long runs with poor equipment at either end. In those scenarios it will show a difference in speed that is measurable. As to interference, electricity is the least likely candidate to cause any link degradation due to inducted interference. The frequencies the two operate at are far too different. However other higher frequency lines may cause interference, e.g in building BDA's for Cellular phones and back hauls for radio communications. This is where there will be a benefit for Cat6. Also when it comes to a home gbit network most switches are a 1 gb dumb switch, and probably wont even come close with every optimization you can find to sustaining more than 1 transfer at 980ish Mbit/secAs to using a cat5e/cat6 patch cable, unless you've got some weird useless nic or have a huge amount of UHF and higher transmitters hanging around, you can use silver satin for a patch cable and attain the same speed (I've done it just to see)Corporate networks however have much more to gain from GB networks. Jumbo frames combined with other optimizations along with multi-pathing make iSCSI a viable option for use as a SAN provided you're switches arent sold at Wal Mart.As to other network considerations such as Jitter and the like which may only be noticeable by streaming sources such as Video over IP Surveillance systems, 100Mbit is preferred for the cameras / encoders as it survives with less problems with extremely sensitive applications. I would like to see more comparisons between soho switches and higher end ones. For example, in my poor experimentation, I cut in half a spool of 1000ft cat5e, (this wire was used later for REAL purposes) and put ice cubes on each end, then plugged both in to an HP 5308 using a 16port 10/100/1000 module. Both machines where fairly high end HP workstations (xw8400 using onboard LSI raid in R0 config with 3 Velociraptors (replaced the stock seagate junker that came with them)Throughput for this was right at 800Mbit/sec from either machine. They both ran Server 2003 R2. I then used the same machines and plugged them into a single Netgear off the shelf switch, and got only 270 at best. I then used regular patch cables on the netgear and poof, 730-800 was attained. File was 8.2GB insize being transferred. If you guys ever get around to benching some Cisco vs HP stuff (at LEAST stackable switches, preferably layer 3) it'd rockCopper 1GB Trunks VS Fiber for example wouldbe another cool one (2 copper = 1 fiber 1G trunk as I remember, so 4 copper members VS 2 1G Fiber to be fair)Site rocks.Thanks for the damn drop down list btw, sucked with out it on the site.[/citation]

hehe someone who's done some real cableing test!
...and knows what he talks about "...horizontal cabling."

 

[crlx]

[citation][nom]Anonymous[/nom]....Next up "Dude why is my DVD drive slower than my hard drive" [/citation]
LOL!

 

[NicNash]

Just wow. I personally think this was a great article. The negative members need to realize this article isnt just for them, and that there are those other than themselves that this greatly helped.

 

[Guest]

Great article, much appreciated even by an old mainframe systems programmer! All these people quibbling over slight inaccuracies and ommissions, have completely missed the point of the article. Just some real world tests to prove his presumptions, and explain to anyone who is interested in a way that is easy to understand and re-test themselves.

 

[Guest]

This is the first site where I read that Cat5 cable supports gigabit. Just about everywhere else says its bollocks, but I think they are the ones who are confused since I connected my gigabit switch tonight with two computers that I care about in terms of data transfer; one using Cat5 (not Cat5e) and one using Cat6. I expected based on what I read to get virtually no gain, but I actually got 104Mb/s transfer (compared to 11Mb/s before). That's about 92Mbps vs 872Mpbs. Reading other comments I'm happy with that 13% loss whilst still using Cat5 cable. I should also note my switch supports 802.3ab which is able to do 1000Mbps over Cat5. So for all those who think they have to upgrade from Cat5, it's not true. You have worse bottle necks than that (most likely) to consider.

 

[Guest]

100Mb/s is 800Mbps and 1Gbps is 1000Mbps. Of course you don't get 10 times faster speed. You should get about 1.2x faster speed, and you did.

 

[Guest]

Awesome article! I learned a lot and had all my questions answered. Thanks!

 

[Ahumado]

Excellent article, it needs updating though to account for fast SSD's

 

[rezwits]

Great article, saved me $100 from buying Cat6