There are actually three different 2 TB limitations that are commonly hit...

• Partition size
• Number of clusters
• SCSI goo

Partition Size Limitation

The partition size is pretty straight forward.  On an MBR (Master Boot Record) disk, the locations where the partition sizes are stored are only 4 bytes long.  Since this is in hexadecimal, the largest value we can stuff in there is all F's.  So the max value would 4,294,967,295 in decimal. 

FF FF FF FFh = 4294967295d

This maximum partition size is not in bytes, it is in number of sectors.  Since currently sectors are limited to 512 bytes, the maximum size ends up being 2 TB.

4,294,967,295 sectors * 512 bytes/sectors = 2,199,023,255,040 bytes or 2TB.

Number of Clusters

The second limitation is harder to spot.  It is a limitation of NTFS.  NTFS is limited to (2^32 -1) clusters....no matter what.  The smallest cluster size possible is 512 bytes (1 sector).  So again the math leaves us at 2,199,023,255,040 or 2TB.

(2^32)-1 = (4,294,967,296)-1 = 4,294,967,295 clusters

4,294,967,295 clusters * 512 bytes/cluster = = 2,199,023,255,040 bytes or 2TB

Here is a quick reference chart to help you see the maximum NTFS size for each cluster size.

Cluster size	NTFS Max Size
512 bytes	2,199,023,255,040 (2TB)
1024 bytes	4,398,046,510,080 (4TB)
2048 bytes	8,796,093,020,160 (8TB)
4096 bytes	17,592,186,040,320 (16TB) Default cluster size
8192 bytes	35,184,372,080,640 (32TB)
16384 bytes	70,368,744,161,280 (64TB)
32768 bytes	140,737,488,322,560 (128TB)
65536 bytes	281,474,976,654,120 (256TB)

Cluster size really depends on your needs.  While 512 is fine if you just have a bunch of tiny files, it isn't as efficient for say a volume with just SQL DBs.  Also, a tiny cluster size can adversely affect VSS.  But that is a topic for another time.

SCSI Goo

This is by far the hardest to understand as it requires some basic SCSI knowledge.  Microsoft Windows operating systems support two different SCSI standards when it comes to reads and writes.  There is a third but it is very old and is mostly just used one tape devices.  So let's just forget about that one and stick to the two that are relevant.

These two standards are Read10/Write10 and Read16/Write16.  This all has to do with the way the CDB (Command Descriptor Block) is structured.

Read10/Write10 - This standard reserves bytes 2-5 to define the LBA (Logical Block Address).  Think of LBA as sector numbers....it makes it easier on your brain.  So we have 4 bytes that can define the addressable sectors.  Just like in the ‘partition size limitation' we are back to dealing with a 4 byte number used to define all the addresses on the drive.

FF FF FF FFh = 4294967295d

And just like before, the above is just the possible number of address (number of sectors).  Multiply by the standard sector size of 512 bytes and we get...

4,294,967,295 sectors * 512 bytes/sectors = 2,199,023,255,040 bytes or 2TB.

What this all means is that when Windows is using the Read10/Write10 standard, then the biggest drive that will be supported is 2TB. 

Read16/Write16 - Sometimes called LBA64 by some vendors, this standard reserves bytes 2-9 to define the LBAs.  That would be 8 bytes, each byte being 8 bits in size.  Now here is where we start getting into some really big numbers.  The approximate size comes out to be around 8ZB (zettabytes).  Here's a quick chart to put it in some perspective.

• 1024KB = 1MB (megabyte)
• 1024MB = 1GB (gigabyte)
• 1024GB = 1TB (terabyte)
• 1024TB = 1PB (petabyte)
• 1024PB = 1EB (exabyte)
• 1024EB = 1ZB (zettabyte)

So it is going to be a while before we have to worry about running into the limitation of Read16/Write16.

Exceeding the limitations

Each limitation has a way of getting around it.  Otherwise we'd still be stuck at 2TB. 

Partition size limitation - There are actually two ways around this.  The first way is to convert your disks to dynamic disks and create volume sets.  This functionality has been around since Windows 2000.  This doesn't really allow you to increase the partition size.  What it does is give you the ability to chain your partitions together to form larger volumes.  So I could use two drives of 2TB and create a volume of roughly 4TB in size.

The second method to bypass the partition size limitation is to use a GPT (Guid Partition Table) configuration.  In Windows 2003 SP1 Microsoft introduced its implementation of the GPT.  A disk configured to be GPT rather than the MBR style would have a 32 sector partition array instead of a tiny 64 byte partition table.

NOTE:  32 sectors is equal to 16,384 bytes

The partitions that can be defined on a GPT disk can be up to 16EB in size.       

Number of clusters - There really isn't a way around this limitation.  NTFS is currently still limited in its number of clusters.  However, you can get past 2TB by making sure your cluster size is larger than the minimum size of 512 bytes. 

It is important to note that for the most part, if you create a FAT partition and then convert it to NTFS the cluster size will come out as 512 bytes.  There are ways around this but most of them require that you know ahead of time that you are going to be converting your FAT partition to NTFS. 

SCSI goo - There isn't anything in Windows you can do to get around the limitation of Read10/Write10 as   Windows is already able to use Read16/Write16.  However, the hardware MUST support it as well.  Windows will query the storage devices and negotiate if Read10/Write10 or Read16/Write16 is to be used.  When in doubt, check with your storage vendor.

Robert Mitchell
Senior Support Escalation Engineer
Microsoft Enterprise Platforms Support 

Implementing networks in support of Failover Clusters

The main consideration when designing Failover Cluster networks is to ensure there is built-in redundancy for cluster communications.  This is typically accomplished by having a minimum of two physical Network Interface Cards (NICs) installed in each node that will be part of the cluster.  These cards must be supported by two separate and distinct buses (e.g.  Two PCI NICs).  Many people think a single multi-port NIC card meets this requirement - it does not as this configuration creates a single point of failure for all cluster communications.   The best configuration would be two multi-port NICs running on separate buses and having fault tolerance implemented by way of NIC Teaming software (provided by 3^rd Party vendors.) and being physically connected to separate network switches.

Note:  NIC Teaming is not supported on iSCSI connections.  Please review the iSCSI Cluster Support: Frequently Asked Questions.  The appropriate fault-tolerant mechanism for iSCSI connectivity would be multi-path software. Please review the Microsoft Multi-path I/O: Frequently Asked Questions.

There are two primary design scenarios when planning for Failover Cluster network connectivity.  In the first scenario (and the most common), all nodes in the cluster are located on the same networks.  In the second scenario, nodes in the cluster are located on separate and distinct routed networks (this is very common in multi-site cluster implementations).  Figure 18 shows an example of the second scenario.

Figure 18:  Multi-site cluster (network connectivity only)

Note:  Even though it is supported to locate cluster nodes on separate, routed networks, it is still supported to connect nodes in a multi-site cluster using stretched Virtual Local Area Networks (VLAN).  This configuration places the nodes on the same network(s).

It is important in any cluster that there are no NICs on the same node that are configured to be on the same subnet.  This is because the cluster network driver uses the subnet to identify networks and will use the first one detected and ignore any other NICs configured on the same subnet on the same node.  The cluster validation process will register a Warning if any network interfaces in a cluster node are configured to be on the same network.  The only possible exception to this would be for iSCSI (Internet Small Computer System Interface) connections.  If iSCSI is implemented in a cluster, and MPIO (Multi-Path Input/Output) is being used for fault-tolerant connections to iSCSI Storage, then it is possible that the network interfaces could be on the same network. In this configuration, the iSCSI network in the Failover Cluster Manager should be configured such that cluster would not use it for any cluster communications.

Note:  Please consult the iSCSI Cluster support: Frequently Asked Question.

As previously mentioned, Windows Server 2008 accommodates cluster nodes being located on separate, routed networks by including a new logic, called an OR logic, when it comes to IP Address resources.  Figure 19 illustrates this.

Figure 19:  IP Address Resource OR logic

When a Network Name resource is configured with an OR dependency on more than one IP Address resource, this means at least one of the IP Address resources must be able to come Online before the Network Name resource can come Online.  Since a Network Name resource can be associated with more than one IP Address, there is a property of a Network Name resource that can be modified so DNS registrations will occur for all of the IP Addresses.  The property is called RegisterAllProvidersIP (See Figure 20).

Figure 20:  Network Name resource properties

Note:  In Figure 20 above, Failover Cluster PowerShell cmdlets were used to access cluster configuration information.  This is new in Windows Server 2008 R2.  For more information, review the TechNet Cmdlet  Reference.

The default registration behavior is to register only the IP Address that can come Online on the node.  Implementing this other behavior by modifying the setting to (1) can assist name resolution in a multi-site cluster scenario.

Note:  Please review KB 947048 for other things to consider when deploying failover cluster nodes on different, routed subnets (multi-site cluster scenario).

While Failover Clusters require a minimum of two NICs to provide reliable cluster communications, there are scenarios where more NICs may be desired and\or required based on the services or applications that are running in the cluster.  One such scenario we already mentioned - iSCSI connectivity to storage.  The other scenario involves Microsoft's virtualization technology - Hyper-V.

The integration of Failover Clustering with Hyper-V was introduced in Windows Server 2008 (RTM) in the form of making Virtual Machines highly available in a cluster by being able to move (Failover) the Virtual Machines between the nodes in the cluster using a process called Quick Migration.  In Windows Server 2008 R2, additional capabilities were introduced including Live Migration and Cluster Shared Volumes (CSV).  These features improved the high availability story for Virtual machines, but also introduced new networking requirements.   The inner workings of Hyper-V networking will not be discussed here.  For more information, please download this whitepaper (http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=3fac6d40-d6b5-4658-bc54-62b925ed7eea). 

The networking requirements in a Hyper-V Cluster supporting Live Migration and using Cluster Shared Volumes (CSV) can add up quickly as illustrated in Figure 21.

Figure 21: Hypothetical Networking Requirements

For more information on Live Migration and Cluster Shared Volumes in Windows Server 2008 R2, visit the Microsoft TechNet site.

Using Cluster Shared Volumes in a Failover Cluster in Windows Server 2008 R2

Hyper-V:  Using Live Migration with Cluster Shared Volumes in Windows Server 2008 R2

In the next segment I will discuss Troubleshooting cluster networking issues (Part 3).  See ya then.

Chuck Timon
Senior Support Escalation Engineer
Microsoft Enterprise Platforms Support

Här har vi samlat ett antal videodemonstrationer gällande Windows Server 2008 och hyper-V. Demonstrationerna är dock på engelska.

Video 1 "Windows Server 2008 med Hyper-V Start"

Video 2  "Windows Server 2008 Hyper-V regler"

Video 3 "Windows Server 2008 Skapa en virtuell maskin i Hyper-V"

Video 4 "Windows Server 2008 Sätta upp ett Hyper-V kluster"

Video 5 "Windows Server 2008 Placera Virtual Machine i ett kluster"

Video 6 "Windows Server 2008 En titt på snabb migrering"

VIDEO 6

VIDEO 5

VIDEO 4

VIDEO 3 

VIDEO 2

VIDEO 1

Citrix har publiserat ett whitepaper "The Citrix Virtualisation Index" läs dokumentet här.