NTFS cluster size = IO size? Performance gains?

[ricno]

In SAN storage you often speak about the amount of IOPS that can be delivered by a certain combination of disk spindles, RAID level, disk RPM, seek times, cache sizes and similar.

The IOPS (IOs per second) should be the amount of IO operations that could be achieved with a certain IO size, that is, how large the individual read and write is. Often it could be 4kB, 8kB or 16, 32, 64 and so on.

So, something I have often been wondering, is the IO size (which is the operation against the low level sectors on the disk) related to the logical file system allocation unit. For example, in Windows the allocation unit / cluster size is in almost all cases 4 kB. Does anyone know if a different cluster size, for example 8 kB, would mean a different number of IOPS against the disk?

(I know all about the cluster size against disk space allocated and larger cluster sizes could mean more wasted space with small files and so on, but I am thinking now about the IO amount and performance issues.)

 

[sminlal]

Generally speaking there's no dependency between the I/O size and NTFS cluster size in terms of performance. The NTFS cluster size affects the size of the file system structures which track where files are on the disk, and it also affects the size of the freespace bitmap. But files themselves are normally stored contiguously, so there's no more effort required to read a 1MB file from the disk whether the cluster size is 4K or 64K. In one case the file header says "the file starts at sector X and takes 256 clusters" an in the other case the headers says "the file starts at sector X and takes 16 clusters". The system will need to perform the same number of reads on the file in either case no matter what the I/O size is. For example, if the I/O size is 16K then it will take 128 reads to get all the data regardless of the cluster size.

In a heavily fragmented file system the cluster size may start to affect performance, but in that case the solution isn't to try optimize the cluster size, instead you need to fix the problem by getting the files defragmented.

On a drive that performs a LOT of file additions/deletions or file extensions then cluster size can have a performance impact because of the number of I/Os required to update the file system metadata (bigger clusters generally = less I/Os). But that's independent of the I/O size used by the application - the I/Os to update the metadata are part of NTFS itself and aren't something that the application performs.

 

[ricno]

Generally speaking there's no dependency between the I/O size and NTFS cluster size in terms of performance.

Thanks a lot for your answer. I have been doing some experiments with IO load testing and have not seen any difference in both performance, and also in the IO size itself.

That is, if using 512 bytes NTFS Cluster and generating 4KB IO read requests - the Performance Monitor reports the average IO read size to be 4 KB, where I suspected it would be 512 byte. If doing 16KB or 32 KB IOs this was also the IO size.

Also if using say 8 KB cluster size and doing smaller IOs (512 byte - 4096 byte) the IO size reported from Perfmon was the expected. I was in the belive that the IO size would be a minimum of the NTFS cluster size, which was not true.

It seems like the NTFS cluster size is just for the logical filesystem and smaller IOs could "hit" inside it, and if the file are contigous a single IO read request could be issued to the disk.

 

[ricno]

Best answer selected by ricno.

 

[superkikim]

That is, if using 512 bytes NTFS Cluster and generating 4KB IO read requests - the Performance Monitor reports the average IO read size to be 4 KB, where I suspected it would be 512 byte. If doing 16KB or 32 KB IOs this was also the IO size.

Well, I guess when you are talking "I/Os" you refer I/Os generated by a benchmark tool like Io Meter ? If it's the case, I guess it is expected, as my understanding is that the purpose of these applications is to generate the correct size of I/Os regardless of the file system beneath them.