problems with larger then 2TB arrays

[dragonsprayer]

this is a quote from the sandman, from my previous thread - can anyone elaborate on this statement:


"off you are going to have issues with arrays over 2TB.
Yes highpoint blows on the high end. "


Why are 3TB or 4TB arrays an issue? we plan hooking 8 hdd in raid 5 with lsi logic card.

thank you for any input!

 

[sandmanwn]

Windows 2K/XP have a 2TB limit on boot partitions not arrays. Thats a slip up on my part.

Windows 2K3 I believe has been patched so it can exceed 2TB partitions. If its not out yet then it should be soon.

Some controllers are stuck at 2TB LUN limitations. Make sure you read the specs carefully for the controller.

I am not aware of Vista partition size limitation.

 

[misry]

Doesn't x64 exceed the 2T limit as well?

 

[dragonsprayer]

thx!

 

[maury73]

Volume, partition and file size limits don't rely on architecture limitations: it's only an OS limit: even an 8 bit MCU can handle 64 bit long long integers (grouping 8 consecutive bytes together), it's only a matter of OS implementation.

 

[dragonsprayer]

can you decipher that in relation to my question or that just a response to the post about 64 bit?

or, i do not understand the relevance of your answer? (or wtf does that mean (translated in noobie code (joke)))

 

[jamesgoddard]

Windows 2K/XP have a 2TB limit on boot partitions not arrays. Thats a slip up on my part.

Windows 2K3 I believe has been patched so it can exceed 2TB partitions. If its not out yet then it should be soon.

Some controllers are stuck at 2TB LUN limitations. Make sure you read the specs carefully for the controller.

I am not aware of Vista partition size limitation.

Wrong, with ANY version of windows (on x86) there is a limit of 2TB on the boot ARRAY, this is because this is the maximum an array with MBR can support. I have a system witn Windows 2003 SP1, and a 5TB boot array, I had to scrap this in the end and create 3 smaller arrays before I could use the space.

You can use an aray with more then 2TB as a data disk, but you need to use GPT arrays, and you CAN NOT BOOT FROM THEM

http://technet2.microsoft.com/WindowsServer/en/library/4b35160a-4e27-4258-9e8b-e2088f8a757a1033.mspx?mfr=true

 

[zenmaster]

There is more discussion here .........................

http://www.microsoft.com/technet/prodtechnol/winxppro/reskit/c13621675.mspx

 

[sandmanwn]

I do believe that is what I said.

2TB limit on boot partitions not arrays.

Anyway, they are working on a larger than 2TB bootable partition for 2003.

@maury73
Yes there are hardware limitations that can limit the array size to 2TB. It is not completely inclusive to the OS. I believe adaptec is still having issues with the 2TB limit. Most newer controller cards will specifically mention "greater than 2TB" in the specs or state the maximum by mentioning something like "32TB LUN support."

 

[jamesgoddard]

I do believe that is what I said.
2TB limit on boot partitions not arrays.

Anyway, they are working on a larger than 2TB bootable partition for 2003.


Wrong. Windows 2003 Server will NEVER be able to boot >2TB on an x86 (or x64) system. This is only possible with Windows Vista or Lonhorn server in systems with EFI compatable BIOS (aka nothing on the market at the mo)

edit - some nice supporting info... http://www.microsoft.com/whdc/device/storage/GPT_FAQ.mspx

 

[sandmanwn]

Thanks you actually found the link for me. I knew I wasnt imagining things.

EFI is not necessary for greater than 2TB boot recognition. Now as soon as I find the link where they are implementing this for W2K3 I'll post it.

5. Is EFI required for a GPT disk?
No. GPT disks are self-identifying. All the information needed to interpret the partitioning scheme of a GPT disk is completely contained in structures in specified locations on the physical media.

6. How big can a GPT disk be?
In theory, a GPT disk can be up to 2^64 logical blocks in length. Logical blocks are commonly 512 bytes in size.

The maximum partition (and disk) size is a function of the operating system version. Windows XP and the original release of Windows Server 2003 have a limit of 2TB per physical disk, including all partitions. For Windows Server 2003 SP1 Windows XP x64 edition, and later versions, the maximum raw partition of 18 exabytes can be supported. (Windows file systems currently are limited to 256 terabytes each.)

 

[dragonsprayer]

For the systems i asked about: we are booting from sb control with 680i chipset raid0 2x raptors. So we will be fine with a 4TB raid????


------------------------------

the 4 tb is video storage - we need a single a storage due to streaming issues with multiple xbox 360's used as media exteneders

system locks and freezes when multiple drives are accessed - its a vista issue.

system must be vista comptible

 

[SomeJoe7777]

EFI is not necessary for greater than 2TB boot recognition. Now as soon as I find the link where they are implementing this for W2K3 I'll post it.

Sorry, but jamesgoddard is correct. EFI is most certainly necessary for boot partitions to reside on a device with > 2^32 blocks (2TB).

The quotes you refer to are correct for large (> 2TB) arrays in general when used as data-only arrays, not boot arrays.

I recently set up a Windows Server 2003 with a 6TB iSCSI array, and learned a few things while I did it. 8)

Here is a summary:

1. The only versions of Windows that support >2 TB arrays at all are Windows XP x64, Windows Server 2003 SP1 (x86 and x64), and Windows Vista (x86 and x64).

2. The only versions of Windows that can use a >2 TB array as a boot device are Windows Vista and the not-yet-released Longhorn server. They can only do this on a system with an EFI BIOS. Any system with a standard PC BIOS cannot boot from a >2 TB array regardless of operating system.

3. To support an array >2 TB, the device must be converted to a GPT disk in Windows Disk Management.

4. To support a device >2 TB, the controller must support some type of extended addressing mechanism (48-bit LBA for IDE/SATA devices, 64-bit LBA / 16-byte CDB for SCSI API-compatible controllers). If the controller doesn't support this, the maximum array size it can support is 2TB.

5. Some controllers will assist you in hooking up a large array to an operating system that doesn't support it (such as Windows XP) by using LUN carving. Using this, a large array appears to the computer as multiple 2TB arrays. Each 2TB virtual array can then be formatted as a standard MBR disk. Of course, not all of your space will be on one volume.

6. The FAT32 file system is not supported on arrays >2TB. Volumes >2TB must be NTFS. NTFS will support up to a 256TB array in its current implementation.

7. It is recommended that >2TB arrays be basic disks unless absolutely necessary to make them dynamic disks. Dynamic disks are only supported on devices that are direct connected or connected on a SAN through a Fibre Channel card or iSCSI hardware card. Dynamic disks are not supported using the Microsoft iSCSI software initiator.

8. Some RAID controllers and SAN units have a hack that is supposed to let them get around the 2TB limit on operating systems that don't support it. To implement it, they increase the sector size from the standard 512 bytes up to as large as 4096 bytes. Using this, they purport to be able to support up to a 16 TB array as an MBR disk. Do not use this mechanism under Windows. Windows specifically does not support any device with a sector size other than 512 bytes. Under Linux OS's, this should work OK.

 

[dragonsprayer]

wow - good stuff
thank you!

 

[Jimmy32]

To anybody thinking about Volums >2TB on clustered Windows Servers follwo this link:

http://technet2.microsoft.com/WindowsServer/en/library/4b35160a-4e27-4258-9e8b-e2088f8a757a1033.mspx?mfr=true

Note what they wrote:

• You cannot use the GPT partitioning style on removable media, or on cluster disks that are connected to shared SCSI or Fibre Channel buses used by the Cluster service.

So, if you are running clusterd (failover cluster) Servers, you will be allways limited to max. partition size 2TB.

 

[Zorg]

Not really related to your discussion but I searched Wiki to understand EFI and at the bottom I saw this:

Security and freedom concerns

One of the stated goals of EFI is to protect hardware vendors "intellectual property"[2]. This raises security concerns and notably makes creating a free software BIOS impossible.

EFI could be used to create a "DRM BIOS", thus letting vendors build computers which limit what the user can do.

SWEET :!:

Edit: link
Extensible Firmware Interface - Wikipedia
http://en.wikipedia.org/wiki/Extensible_Firmware_Interface

 

[roallin]

Why would you want a boot partition larger than 2 TB? Why would you want one even close to that size?

 

[jamesgoddard]

Why would you want a boot partition larger than 2 TB? Why would you want one even close to that size?

Can I quote you on that in 10 years time?

 

[dragonsprayer]

Why would you want a boot partition larger than 2 TB? Why would you want one even close to that size?

we have had problems in past with multiple drives and multiple media extenders. it appears if the movies are stored on single drive or in this case a single large partition of 8 x 500gb drives then we have less problems with lag for lack of a better word - when watching 3 different movies using xbox 360's as a media extender on 3 different tv's it seem to work better with a single drive.

 

[darkangelism]

i would use a seperate raid 1 for your OS and use a raid 5 for your data, 8 drives raid 0 for 4TB is just asking for failure and losing all your data, raid 5 could at least handle single drive failure, but still wont replace a backup...speaking of which what are you using to back that up?

 

[Casper42]

Why would you want a boot partition larger than 2 TB? Why would you want one even close to that size?

Its not just a matter of Partitions being over 2TB. Its a matter of the Volume being over 2TB that freaks out Windows for boot.

I recently helped a friend upgrade his rig and we went with 5 x WD6400AAKS drives which would give us around 2.3 TB. We planned to carve out 400GB or so and use this for OS and Apps and Games and then use the remaining 1.9TB as a large MBR storage partition.

However since he was on a 680i, the RAID setup doesnt offer you the ability to either create multiple volumes on a single RAID Group or do Intel's "Matrix RAID".

This means Windows Vista x64 boot DVD sees it as 1 HDD and instead of showing 2.3TB it shows like 320GB (the Wrap around mentioned in this thread previously).
We installed to that partition hoping the other 2TB would be listed as unused after we get into the OS and get the latest nForce drivers on there but the other 2TB never shows up.

So if your on an nForece board (680, 780, 790) beware of large single arrays and booting from them AT ALL.
You would be better off doing a small boot drive or RAID0 with something like Raptors and then save the uber arrays for storage.

I almost told him we should just hook up a 40GB IDE Drive and boot from that and then create a 400GB D: and a 1.9TB E: and just put all his apps and games on D: and use C: for ONLY the OS. Somehow I didnt think he would go for that so we ended up doing a 1.2TB RAID0 with 2 drives and then a 1.2TB Using RAID5. Problem is even with all the latest games and apps on C:, hes never going to fill that drive and now D: is limited to 1.2 when it should have been 1.9 or so.

I so wished he had an Intel board at that moment.
But then he couldnt run his SLI setup.
So once again the fighting of the big companies screws the consumer.

 

[darcehx]

I am kind of new on this, but I am interested on finding out the reason why is not possible to boot to =>2 TB partitions. Something that I think is that this is also somehow related to the hardware since on my motherboard (DX38BT) with Intel RAID controller ich9r, this chipset itslef is able to support 2- 256tb volumes but only for data not to be bootable. So, it doesnt seem to me that is only related to Operating System implementation.

http://www.intel.com/support/chipsets/imsm/sb/cs-022304.htm

The only platform I am aware that is able to boot to a partition like this is Itanium which is truely 64 b architecture. So seems like hardware has to do something to do with this too.

Any guidance or explanation?

 

[SomeJoe7777]

I am kind of new on this, but I am interested on finding out the reason why is not possible to boot to =>2 TB partitions. Something that I think is that this is also somehow related to the hardware since on my motherboard (DX38BT) with Intel RAID controller ich9r, this chipset itslef is able to support 2- 256tb volumes but only for data not to be bootable. So, it doesnt seem to me that is only related to Operating System implementation.

http://www.intel.com/support/chipsets/imsm/sb/cs-022304.htm

The only platform I am aware that is able to boot to a partition like this is Itanium which is truely 64 b architecture. So seems like hardware has to do something to do with this too.

Any guidance or explanation?

This was already discussed in this thread.

First, the limitation is on the boot device, not the boot partition. The boot device cannot be larger than 2TB, regardless of the partitions that are on it.

The reason for that is that the PC standard BIOS cannot address a device that is larger than 2TB, because the PC BIOS is designed as a 32-bit system. The BIOS can load the OS files on a device up to the 2^32 block, after that, the BIOS has no way to address blocks that are further out on the device. So if ntkernel or ntldr (for Windows) resides on a block past 2^32 (which would be legal on a device that's bigger than 2TB), the BIOS cannot load that file, thus the machine won't boot.

Now, you might think that if the boot partition resides completely within the first 2TB that you'd be OK, but that's not the case. When the BIOS goes to access the device, it has to know its size, and if the size is larger than 2TB, it can't address it.

The Itanium machines have an EFI BIOS, a new and completely different BIOS that has the ability to address blocks up to 2^64 on the boot device. Thus, it can boot to a device larger than 2TB.

After everything was cleared up in this thread, it was stated that a >2TB boot device is not supported on normal PC hardware, regardless of operating system. (See #2 in my first post in the thread).