Walking the line between being descriptive and providing all the details, and avoiding a wall of text is tough. So, I've divided it in parts and you can skip the parts you think are irrelevant.
Contents:
Problem: short description
Hardware
Software
Partition Info
Raid Controller
Process of Error Finding
Things it's probably not
Problem: short description
I have 10x 2TB drives in a raid6 configuration (actual size roughly 14.5TB excluding parity drives). The raid controller initializes the logical drive fine, however a (slow) format in windows appears to simply unallocate the drive upon OS restart, whereas a quick format appears to work for a while (as in, a few reboots) then reset to raw unformatted partition (after 3.5TB was copied to it).
Hardware:
Promise SuperTrak EX16350 (16x SATA300 RAID, 128MB PCI-e x8) - Oddly, the controller's BIOS claims it has 256MB memory
Samsung EcoGreen F3EG, 2TB - x10, in a raid6 (roughly 14.5TB of actual storage) - connected to promise controller
Western Digital Caviar Green WD10EACS, 1TB - x6, in a raid6 (roughly 3.6TB of actual storage) - connected to promise controller
OCZ Vertex 2 SOCZSSD2-2VTXE120G 120GB - connected to mobo
HDD 320GB 7200RPM S-ATA300 Seagate 7200.10 16MB Cache - x2, connected to mobo, soft (in windows) mirror raid
MSI DKA790GX
AMD Phenom X4 9950
OCZ Platinum Dual Channel OCZ2P10004GK - 2GB, x4
Silverstone Strider PSU 750W (SST-ST75F-P) - powers everything else
POWER SUPPLY YESICO 560W SilentCool w/modular cable - 'always on' - powers 5 drives: 4 2TB drives and 1 1TB drive
Software:
Windows 7 Ultimate x64
Partition Info
The computer has 4 partitions even without the new 14.5TB raid6 - all are NTFS and Healthy:
[SSD, MBR] C: - Simple, Basic - Boot, Page File, Crash Dump, Primary Partition
[SSD, MBR] System Reserved (no drive letter) - Simple, Basic - System, Active, Primary Partition
[2x 320GB, MBR] D: - Mirror, Dynamic
[3.6TB raid6, GPT] T: - Simple, Basic - Primary Partition
[14.5TB raid6, GPT] S: - Simple, Basic RAW - Primary Partition(... sometimes; Before it was reverting back to unallocated)
Raid Controller
The raid controller goes through initialization fine (I set sector size to max of 4096 and stripe size to 64k), I guess it takes roughly 32 hours. One drive took roughly 8 hours to check (full drive read) using samsung's estool and there are 2, 4, 4 drives over 3 connections to the raid card so that's what you'd expect really. All 10 hard disks and the logical drive check out as OK in the controller bios. The hard disks also have the same identical firmware. Aside from the logical drive initialization there is no background activity. The raid card currently also holds a 6x 1TB raid6 which is working fine (unfortunately it's also filled to the brim). For years I've had this raid6 running in conjunction with a 4x 320GB 0+1 raid on the raid card with no problems.
Process of Error Finding
First I allocated the drive in the raid controller BIOS. Fool that I was, I did not wait for the logical drive to be initialized. After a first (slow) format in windows, and restarting, the drive showed as 22% initialized in the raid controller bios and subsequently the drive was back to unallocated in windows. Now I waited for the drive initialization to be complete in the raid bios and proceeded to do another (slow) format in windows. Because the stripe size is 64k, I picked an "Allocation Unit Size" of 64k. Volume name of "Storage" (same as T: ), file system of NTFS (no other choice given, anyway). Quick format ticked off. "Enable file and folder compression" was unavailable (because the allocation unit size wasn't default, I assume) - and undesired. This second format did the same as the previous: the drive was unallocated (computer management -> storage -> drive management) after an OS restart.
So I removed the logical drive in the controller BIOS, reallocated it (same options as before - 4k sector size (options: 512 to 4096), 64k stripe size (options: 32k, 64k, 128k)) and waited some 50+ hours (far more than the estimated 32 hours required) before rebooting. After reboot the drive showed as fully initialized in the raid bios. I did another slow format in windows. The drive (apparently) works just fine before rebooting. I copied over a few gigs of music - directory listing appeared fully functioning and music playback was no problem. After another reboot the drive was once again unallocated. I did a quick format (same settings still), copied over some music, tested playback, rebooted. The drive appeared to be fully functioning even after several reboots. Next I copied over 3.5TB of data. Suddenly some directories became unavailable. You could see most directories, but (I think) not inside most directories with files in it. The drive's properties showed 3.5TB in use, however selecting all folders in the root (excluding of course an empty recycle bin directory and the system volume information directory) showed a mere 300GB of actual files. After a reboot this drive reverted to a RAW filesystem status in computer management - but not unallocated.
Things it's probably not
At this point I'm stumped.
- NTFS is limited to 256TB with 64k clusters (aka, I think, Allocation Unit Size). Even with 4k clusters it's just under 16TB which is still far more than the 14.5TB it actually is.
- Windows 7 versions do have a ram limitation but no drive size limitation that I could find.
- Before I did all this, I contacted the promise support. Their tech guy assured me that while there might be size limitations of a 32bit OS, a 64bit OS could have a virtually endless size (responding to my question whether the raid controller card had a drive size limit, given that it's handling (including both parity drives) roughly 18.1TB of data).
- I have read (in a 4 year old forum post) that mixing MBR with GPT results in a maximum of 4 primary partitions. Now I'm guessing this is circumvented either by newer technology, or one of several other reasons. I have added a 2TB drive directly, and while I can't remember whether this was an MBR or GPT drive, it worked fine and when it was removed and the 14.5TB drive was added in its stead the partitions and drives remained the same number so I don't think this can be it either.
Contents:
Problem: short description
Hardware
Software
Partition Info
Raid Controller
Process of Error Finding
Things it's probably not
Problem: short description
I have 10x 2TB drives in a raid6 configuration (actual size roughly 14.5TB excluding parity drives). The raid controller initializes the logical drive fine, however a (slow) format in windows appears to simply unallocate the drive upon OS restart, whereas a quick format appears to work for a while (as in, a few reboots) then reset to raw unformatted partition (after 3.5TB was copied to it).
Hardware:
Promise SuperTrak EX16350 (16x SATA300 RAID, 128MB PCI-e x8) - Oddly, the controller's BIOS claims it has 256MB memory
Samsung EcoGreen F3EG, 2TB - x10, in a raid6 (roughly 14.5TB of actual storage) - connected to promise controller
Western Digital Caviar Green WD10EACS, 1TB - x6, in a raid6 (roughly 3.6TB of actual storage) - connected to promise controller
OCZ Vertex 2 SOCZSSD2-2VTXE120G 120GB - connected to mobo
HDD 320GB 7200RPM S-ATA300 Seagate 7200.10 16MB Cache - x2, connected to mobo, soft (in windows) mirror raid
MSI DKA790GX
AMD Phenom X4 9950
OCZ Platinum Dual Channel OCZ2P10004GK - 2GB, x4
Silverstone Strider PSU 750W (SST-ST75F-P) - powers everything else
POWER SUPPLY YESICO 560W SilentCool w/modular cable - 'always on' - powers 5 drives: 4 2TB drives and 1 1TB drive
Software:
Windows 7 Ultimate x64
Partition Info
The computer has 4 partitions even without the new 14.5TB raid6 - all are NTFS and Healthy:
[SSD, MBR] C: - Simple, Basic - Boot, Page File, Crash Dump, Primary Partition
[SSD, MBR] System Reserved (no drive letter) - Simple, Basic - System, Active, Primary Partition
[2x 320GB, MBR] D: - Mirror, Dynamic
[3.6TB raid6, GPT] T: - Simple, Basic - Primary Partition
[14.5TB raid6, GPT] S: - Simple, Basic RAW - Primary Partition(... sometimes; Before it was reverting back to unallocated)
Raid Controller
The raid controller goes through initialization fine (I set sector size to max of 4096 and stripe size to 64k), I guess it takes roughly 32 hours. One drive took roughly 8 hours to check (full drive read) using samsung's estool and there are 2, 4, 4 drives over 3 connections to the raid card so that's what you'd expect really. All 10 hard disks and the logical drive check out as OK in the controller bios. The hard disks also have the same identical firmware. Aside from the logical drive initialization there is no background activity. The raid card currently also holds a 6x 1TB raid6 which is working fine (unfortunately it's also filled to the brim). For years I've had this raid6 running in conjunction with a 4x 320GB 0+1 raid on the raid card with no problems.
Process of Error Finding
First I allocated the drive in the raid controller BIOS. Fool that I was, I did not wait for the logical drive to be initialized. After a first (slow) format in windows, and restarting, the drive showed as 22% initialized in the raid controller bios and subsequently the drive was back to unallocated in windows. Now I waited for the drive initialization to be complete in the raid bios and proceeded to do another (slow) format in windows. Because the stripe size is 64k, I picked an "Allocation Unit Size" of 64k. Volume name of "Storage" (same as T: ), file system of NTFS (no other choice given, anyway). Quick format ticked off. "Enable file and folder compression" was unavailable (because the allocation unit size wasn't default, I assume) - and undesired. This second format did the same as the previous: the drive was unallocated (computer management -> storage -> drive management) after an OS restart.
So I removed the logical drive in the controller BIOS, reallocated it (same options as before - 4k sector size (options: 512 to 4096), 64k stripe size (options: 32k, 64k, 128k)) and waited some 50+ hours (far more than the estimated 32 hours required) before rebooting. After reboot the drive showed as fully initialized in the raid bios. I did another slow format in windows. The drive (apparently) works just fine before rebooting. I copied over a few gigs of music - directory listing appeared fully functioning and music playback was no problem. After another reboot the drive was once again unallocated. I did a quick format (same settings still), copied over some music, tested playback, rebooted. The drive appeared to be fully functioning even after several reboots. Next I copied over 3.5TB of data. Suddenly some directories became unavailable. You could see most directories, but (I think) not inside most directories with files in it. The drive's properties showed 3.5TB in use, however selecting all folders in the root (excluding of course an empty recycle bin directory and the system volume information directory) showed a mere 300GB of actual files. After a reboot this drive reverted to a RAW filesystem status in computer management - but not unallocated.
Things it's probably not
At this point I'm stumped.
- NTFS is limited to 256TB with 64k clusters (aka, I think, Allocation Unit Size). Even with 4k clusters it's just under 16TB which is still far more than the 14.5TB it actually is.
- Windows 7 versions do have a ram limitation but no drive size limitation that I could find.
- Before I did all this, I contacted the promise support. Their tech guy assured me that while there might be size limitations of a 32bit OS, a 64bit OS could have a virtually endless size (responding to my question whether the raid controller card had a drive size limit, given that it's handling (including both parity drives) roughly 18.1TB of data).
- I have read (in a 4 year old forum post) that mixing MBR with GPT results in a maximum of 4 primary partitions. Now I'm guessing this is circumvented either by newer technology, or one of several other reasons. I have added a 2TB drive directly, and while I can't remember whether this was an MBR or GPT drive, it worked fine and when it was removed and the 14.5TB drive was added in its stead the partitions and drives remained the same number so I don't think this can be it either.