RAID 0 slows down after being filled with data.

[chriscw]

We use RAID 0 arrays to store data captured by our high speed microfilm scanners. When first installed and formatted our RAID arrays typically achieve write speeds of about 96 MB/s which is what we need as the scanners can generate up to 80 GB/s of data.

The systems hosting the arrays are Quad Core Intel systems with Intel x58 chipsets

As the disk fills up the speed starts to drop off so that when about 90% full it has dropped to 40 GB/s, due to the way the data is written this is not wholly unexpected. If we then delete all the data to and re test the drive the write speed is still only about 40 GB/s. This only recovers if we then reformat the drive.

Does anyone know what causes this and whether there is a work round.

 

[chunkymonster]

What RAID controller are you using to host the arrays? A dedicated hardware controller? Or the onboard ICH10R controller? Do you have the latest chipset drivers? Do you have the latest BIOS versions? Did you install the RAID management utility?

The array only recovers when you reformat due to the fact that it is a clean wipe of the drive as opposed to just erasing the data which the array just marks those sectors as over writable.

 

[chriscw]

We are using the onboard ICH10R controller, I do not think we have updated the Bios on any of the Motherboards. We have repeated the test on two other systems both of which recover to close to their original performance after just deleting the data. Because all that deleting does is mark the data as over-writable I would expect a small slow down but not the 50% that the 'bad' system is showing...

We do not have the RAID Management software installed on the Windows 7 systems, just the drivers and we are using hardware RAID. The motheboards we have are all ASUS P6T ones.

 

[sminlal]

The physical geometry of a disk drive is a series of concentric magnetic "tracks" of data on one or more platters. As you fill the drive, you start with the outermost tracks and move toward the inner ones. Since the inner tracks are shorter, they hold less data - and since the entire platter spins at the same speed this means the transfer rate gets slower as you move toward the inner tracks.

Unfortunately, just deleting the files from the disk doesn't "reset" where the next files will be written. They could well be still writing to the as-yet-unused inner tracks. I don't know of any way of resetting this short of formatting the drive.

There are two possible solutions that I can see:

1) Partition your RAID array with a single smaller partition that doesn't fill up the whole drive so that you're using only the outer tracks on the disks. This will reduce capacity, but eliminate any chance of using the inner tracks which don't perform well enough for you.

2) Buy new disks which have adequate performance throughout their entire range.

 

[chriscw]

Thanks Sminlal,

That puts the facts as we know them and the symptoms we are observing quite nicely. I am still slightly surprised by just how much the thing slows down but otherwise quite a masterful explanation.

 

[sminlal]

Follow-up: you can easily see the effect of the slowing speed across the different tracks of the drive when you look at graphs of transfer rate vs. head position such as this one (note the blue line on the graph):

If you can find a disk whose worst transfer rate still fits your needs, you'll be set.

 

[Guest]

To Sminlal. First of all thank you, at last an explanation that I understand about this issue.
Now I have a question of my own.
You wrote: "Unfortunately, just deleting the files from the disk doesn't "reset" where the next files will be written. They could well be still writing to the as-yet-unused inner tracks. I don't know of any way of resetting this short of formatting the drive."

My question might be utterly stupid, but what about defrag of the disk? Wouldn't this reset where the next files will be writen?

 

[sub mesa]

If the filesystem is empty enough to allow relocating; yes. You need ~30% empty to let defrag do its work.

Another method would be short stroking; create a partition in the fastest area of an HDD (or RAID0 array of two HDDs). Now create a second much larger partition for the rest.

So you have:

C-partition: fastest performance (100GB large)
D: partition: slower performance (the rest)

This technique is known as short stroking; it forces data written to partition C to be on the fastest spot on your HDD. The downside is less space for C, and the advantage doesn't work if you use C and D at the same time.

If you would run a benchmark on the C partition, you would see a horizontal line, instead of a line that slowly decays, such as in the picture sminlal posted.

 

[sminlal]

My question might be utterly stupid, but what about defrag of the disk? Wouldn't this reset where the next files will be writen?

Well I have to emphasize that I don't actually know what the algorithm is for determining where the next file will be written, so I can't really tell you whether defragging the disk will reset it. I think the only way to know for sure would be to do before-and-after performance tests. Even then, observing that it does (or doesn't) do it after one defrag is no guarantee that it will behave the same way for other defrags...

 

[MRFS]

The read/write heads try to maintain the same or similar
recording density from outermost to innermost tracks:

because circumference diminishes from outer to inner tracks,
the amount of data on any given track is directly proportional
to its circumference (PI x diameter).

Consequently, as correctly explained above, when a HDD fills up,
both READs and WRITEs slow down because the armature
is reaching further inwards towards the spindle axis.


Conversely, if the same recording RATE were maintained
during all WRITEs, binary digits would become closer and closer
as the armature moves from outer to innermost tracks.

(You can simulate this roughly with a spray can and a spinning frisbee,
or use a string of cheap plastic pearls, and create 2 circles of
different sizes with those pearls e.g. one for your wrist and
one for your neck -- each discrete pearl is a binary digit


The media and electronics are simply not designed
to accommodate such a widely varying recording density:
so, the solution is to maintain the same or similar
recording density, which necessarily means a HDD
will slow down as it fills up.

See HDTune graphs, for a clear illustration.


MRFS

 

[chriscw]

As far as defragging goes for what it is worth we have found that using the Defrag built in to windows on a RAID 0 array normally dramatically slows it down. We have observed this on every system where we have tried it. I suspect that a more proffessional package like Biuzzsaw or the full commercial version of speed disk may do better.

Defrag on a no RAID disk does seem to help in most cases.