Better Performance Question

[rich1051414]

So I have just finished building and setting up my home NAS(16TB, 8TB effective due to redundancy, Raid 10), which is for file security and easy access from multiple computers in the house, and the network is running on a Gigabit network, and large files peak at 112MB/ps which sounds spot on to me.
The problem is it still feels a bit slow, especially for Terabyte level transfers.

I am assuming this is the best I am going to do on a gigabit network, so my question is, what kind of cost and work would I need to dedicate to up that to 10 Gb. Would this be feasible on Cat 7 shielded(my current networking), or would I be better off looking at fiber?

 

[jstear]

If you need faster, fiber would be your option, yes. This needs asked though, do you really need those kind of speeds for home use?

 

[jstear]

What NAS do you have and do you have any switches between?

 

[bill001g]

You can use cat6a you would only use shielded cable if you have problems. If you are going fairly short distances you can use normal cat6 it says 55 meters but there is no guarantee.

The cabling is fairly inexpensive compared to the ends devices. First you would need 10g ports on the NAS and the NAS would actually have to be designed to use the ports. This generally means you are using high rpm drives. This is why you see commercial nas devices use 10k and 15k rpm drives.

Next you need a 10g port on your pc/server. Generally to make this work well you will have to put it in one of the PCI you normally put video cards in. Even when you do this you need to be sure there is enough total bandwidth between the processor and the interfaces on the motherboard to drive all the equipment in the machine...ie the local hard drives and the video as well as the 10g card. This is one of the things that make a xeon processor different than say most i7...although the latest i7 has more abilities.

When you start looking at 10g you are in effect designing a server because there are a lot of bottlenecks in a normal desktop that have no impact but do when you are trying to run at 10g.


Note if you actually have cat7 cable now be very very sure both ends of the shield are properly grounded. It takes special jacks and plugs. Running shielded cable incorrectly installed is actually worse then running unshielded cable because the shield act as a antenna receiving interference and has no place to dump it.

 

[gbb0330]

So I have just finished building and setting up my home NAS(16TB, 8TB effective due to redundancy, Raid 10), which is for file security and easy access from multiple computers in the house, and the network is running on a Gigabit network, and large files peak at 112MB/ps which sounds spot on to me.
The problem is it still feels a bit slow, especially for Terabyte level transfers.

I am assuming this is the best I am going to do on a gigabit network, so my question is, what kind of cost and work would I need to dedicate to up that to 10 Gb. Would this be feasible on Cat 7 shielded(my current networking), or would I be better off looking at fiber?

are you sure the network is the problem, with a lot of NAS devices, especially the ones using mechanical drives (I assume you have those, because 16 TB SSD will cost a fortune), you will hit the hard drive's data transfer limit before you hit the gigabit network limit.
what kind of hard drives are in the nas?

 

[jstear]

So I have just finished building and setting up my home NAS(16TB, 8TB effective due to redundancy, Raid 10), which is for file security and easy access from multiple computers in the house, and the network is running on a Gigabit network, and large files peak at 112MB/ps which sounds spot on to me.
The problem is it still feels a bit slow, especially for Terabyte level transfers.

I am assuming this is the best I am going to do on a gigabit network, so my question is, what kind of cost and work would I need to dedicate to up that to 10 Gb. Would this be feasible on Cat 7 shielded(my current networking), or would I be better off looking at fiber?

are you sure the network is the problem, with a lot of NAS devices, especially the ones using mechanical drives (I assume you have those, because 16 TB SSD will cost a fortune), you will hit the hard drive's data transfer limit before you hit the gigabit network limit.
what kind of hard drives are in the nas?

This is where I was eventually going. Your NAS is probably going to be your bottleneck, not your network.

 

[Kewlx25]

If I had the option, I would go fiber only because of the protection is gives of not allow surges to nuke your precious file server.

 

[rich1051414]

So I have just finished building and setting up my home NAS(16TB, 8TB effective due to redundancy, Raid 10), which is for file security and easy access from multiple computers in the house, and the network is running on a Gigabit network, and large files peak at 112MB/ps which sounds spot on to me.
The problem is it still feels a bit slow, especially for Terabyte level transfers.

I am assuming this is the best I am going to do on a gigabit network, so my question is, what kind of cost and work would I need to dedicate to up that to 10 Gb. Would this be feasible on Cat 7 shielded(my current networking), or would I be better off looking at fiber?

are you sure the network is the problem, with a lot of NAS devices, especially the ones using mechanical drives (I assume you have those, because 16 TB SSD will cost a fortune), you will hit the hard drive's data transfer limit before you hit the gigabit network limit.
what kind of hard drives are in the nas?

It is my own, nothing prebuilt, as this is not the way I do things And more than anything, I am asking about possible solution paths and the kind of expense I would be looking at as well as the pitfalls I should expect by going that route.

As far as people being concerned about drives being the bottleneck, its not even close. I am currently capping out my 1gigabit network and the drives are not even running at 1/2 of their potential speed(or more if I expand later).
The drives are configured as to be delivering twice the performance of the performance of a single drive(4 in raid 10), and gigabit caps to 1000 / 8, which is 125MB/s. Speed bottlenecks in the drive itself on gigabit may be a problem when repurposing old drives, but not on modern mechanicals. My raid array, when tested locally, have a sustained write speed of 310 MB/s(10GB/s for cached reads), and perform at 112 MB/s over the network. I do know I MAY get an extra 10MB/s out of using proper nics, and possibly slightly better performance for multiple small files, but that would be ridiculous

With that said, I do a lot of moving of files which need protection from loss and access to from multiple devices in my house, so my need for a nas is obvious, and the bottleneck will only grow as time goes on, and it is already limiting my productivity, but I am content FOR NOW while I gather all the information I can until I do decide to upgrade the network. I did think ahead the last time, and is wired with Cat 7 shielded(the newest iteration when I strung it). My house is not THAT large, but I have heard stories about 10 gig on copper being unreliable, so I don't want to waste my time going that path if it will actually be a waste of my time and money.

 

[gbb0330]

So I have just finished building and setting up my home NAS(16TB, 8TB effective due to redundancy, Raid 10), which is for file security and easy access from multiple computers in the house, and the network is running on a Gigabit network, and large files peak at 112MB/ps which sounds spot on to me.
The problem is it still feels a bit slow, especially for Terabyte level transfers.

I am assuming this is the best I am going to do on a gigabit network, so my question is, what kind of cost and work would I need to dedicate to up that to 10 Gb. Would this be feasible on Cat 7 shielded(my current networking), or would I be better off looking at fiber?

are you sure the network is the problem, with a lot of NAS devices, especially the ones using mechanical drives (I assume you have those, because 16 TB SSD will cost a fortune), you will hit the hard drive's data transfer limit before you hit the gigabit network limit.
what kind of hard drives are in the nas?

It is my own, nothing prebuilt, as this is not the way I do things And more than anything, I am asking about possible solution paths and the kind of expense I would be looking at as well as the pitfalls I should expect by going that route.

As far as people being concerned about drives being the bottleneck, its not even close. I am currently capping out my 1gigabit network and the drives are not even running at 1/2 of their potential speed(or more if I expand later).
The drives are configured as to be delivering twice the performance of the performance of a single drive(4 in raid 10), and gigabit caps to 1000 / 8, which is 125MB/s. Speed bottlenecks in the drive itself on gigabit may be a problem when repurposing old drives, but not on modern mechanicals. My raid array, when tested locally, have a sustained write speed of 310 MB/s(10GB/s for cached reads), and perform at 112 MB/s over the network. I do know I MAY get an extra 10MB/s out of using proper nics, and possibly slightly better performance for multiple small files, but that would be ridiculous

With that said, I do a lot of moving of files which need protection from loss and access to from multiple devices in my house, so my need for a nas is obvious, and the bottleneck will only grow as time goes on, and it is already limiting my productivity, but I am content FOR NOW while I gather all the information I can until I do decide to upgrade the network. I did think ahead the last time, and is wired with Cat 7 shielded(the newest iteration when I strung it). My house is not THAT large, but I have heard stories about 10 gig on copper being unreliable, so I don't want to waste my time going that path if it will actually be a waste of my time and money.

Really sounds like local storage is the best option for your type of work. backup to the NAS overnight . Theoretically CAT 7 supports 10 gigabit, never seen it in action, but I assume it will get close to 10. 10 gigabit NICs are also pretty expensive, switches ain't cheap either, so you are looking at about 2-3 thousand at least. this in not the kind of technology you see frequently. i've been working in IT for over 15 years, never seen a 10 gigabit network at home, or small to medium business.

 

[rich1051414]

So I have just finished building and setting up my home NAS(16TB, 8TB effective due to redundancy, Raid 10), which is for file security and easy access from multiple computers in the house, and the network is running on a Gigabit network, and large files peak at 112MB/ps which sounds spot on to me.
The problem is it still feels a bit slow, especially for Terabyte level transfers.

I am assuming this is the best I am going to do on a gigabit network, so my question is, what kind of cost and work would I need to dedicate to up that to 10 Gb. Would this be feasible on Cat 7 shielded(my current networking), or would I be better off looking at fiber?

are you sure the network is the problem, with a lot of NAS devices, especially the ones using mechanical drives (I assume you have those, because 16 TB SSD will cost a fortune), you will hit the hard drive's data transfer limit before you hit the gigabit network limit.
what kind of hard drives are in the nas?

It is my own, nothing prebuilt, as this is not the way I do things And more than anything, I am asking about possible solution paths and the kind of expense I would be looking at as well as the pitfalls I should expect by going that route.

As far as people being concerned about drives being the bottleneck, its not even close. I am currently capping out my 1gigabit network and the drives are not even running at 1/2 of their potential speed(or more if I expand later).
The drives are configured as to be delivering twice the performance of the performance of a single drive(4 in raid 10), and gigabit caps to 1000 / 8, which is 125MB/s. Speed bottlenecks in the drive itself on gigabit may be a problem when repurposing old drives, but not on modern mechanicals. My raid array, when tested locally, have a sustained write speed of 310 MB/s(10GB/s for cached reads), and perform at 112 MB/s over the network. I do know I MAY get an extra 10MB/s out of using proper nics, and possibly slightly better performance for multiple small files, but that would be ridiculous

With that said, I do a lot of moving of files which need protection from loss and access to from multiple devices in my house, so my need for a nas is obvious, and the bottleneck will only grow as time goes on, and it is already limiting my productivity, but I am content FOR NOW while I gather all the information I can until I do decide to upgrade the network. I did think ahead the last time, and is wired with Cat 7 shielded(the newest iteration when I strung it). My house is not THAT large, but I have heard stories about 10 gig on copper being unreliable, so I don't want to waste my time going that path if it will actually be a waste of my time and money.

Really sounds like local storage is the best option for your type of work. backup to the NAS overnight . Theoretically CAT 7 supports 10 gigabit, never seen it in action, but I assume it will get close to 10. 10 gigabit NICs are also pretty expensive, switches ain't cheap either, so you are looking at about 2-3 thousand at least. this in not the kind of technology you see frequently. i've been working in IT for over 15 years, never seen a 10 gigabit network at home, or small to medium business.

You are probably right, and I am getting too impatient for technology to progress. We have been sitting on 1gig networking for so long it is utterly fascinating we are still using it. You already described what I am doing now, as using the nas as a local drive does cause a noticeable performance hit, but manageable, seems to not be ideal and limiting for there not to be any advancement on this front on the consumer level.
Ah well, I'll just sit back and wait some more

It does make me wonder why some people would run a nas on Raid 0, and using 0 at ALL on a home nas with a modern drive seem completely pointless, unless accessing from multiple devices at the same time...