NetApp Usable Space Calculator ver 1.0
For people who are looking at a quick way to calculate the usable space of a NetApp Filer, I have developed this tool. It is a stand alone executable (Approx 2 MB) which will run on Windows platforms. No Installation needed. I understand there may be bugs in the tool. I am ready for feedback and would like to improve this tool. For now please download and use and let me know. Please use this tool as a estimate and this tool is supplied without any warranty. It may not report accurate sizes.
Aug 8 2013 - NEW VERSION is 2.0 Refer to my
post here (2.5MB)
Many people have blogged about the usable calculation formula. I am not going to repeat the steps here. Instead I will give some pointers to those blogs. Below info was used in developing this calculator. Feedback recommended.
STEP1: Calculate the disk size first
Basically if you go to bestbuy and pick a 1TB drive, you actually think that you store 1TB of pictures and music. Thats not the truth. Storage hardware is using the base 10 system and software is using the base 2 system. So no storage is actually lost, it is just a question of how the information is represented.
Hard disk manufacturers assume / Base 10 System - Kilo = 10 power 3
File system assumes / Base 2 System: Kilo = 2 power 10 = 1024
| KB | 210 | 1,024 bytes |
| MB | 220 | 1,048,576 bytes |
| GB | 230 | 1,073,741,824 bytes |
| TB | 240 | 1,099,511,627,776 bytes |
So a 1 TB hard disk is actually
1000,000,000 bytes/ 1,073,741,824 = .9313 x 1000 = 931.3 GB
In a similar way
750 GB hard disk is actually
750,000,000/1,073,741,824 = .6986 x 1000 = 698.6 GB
Base 10 Capacity
|
Base 2 Capacity
|
Right Sized Capacity
|
72GB FC
|
69 GB
|
67.9 GB
|
144GB FC
|
137.9GB
|
135.8 GB
|
300GB SAS
|
279.4 GB
|
272 GB
|
450GB SAS
|
419.1 GB
|
415 GB
|
600GB SAS
|
558.8 GB
|
553.26 GB
|
500GB SATA
|
465.7 GB
|
413.9 GB
|
750 GB SATA
|
698.6 GB
|
620.9 GB
|
1TB SATA
|
931.3 GB
|
827.8 GB
|
2TB SATA
|
1,862.6 GB
|
1,655.7 GB
|
3TB SATA
|
2,794 GB
|
2,483.5 GB
|
Now from the above table you are clear that how a 1TB hard drive has come finally to 931.3 GB. But how did we get the number Right sized capacity 827.8..This is called the checksum calculation which we will see it next.
STEP 2 : Check Sum Calculation
From 931.3 multiply x 8/9 and you will get 827.8 GB. Simple..Why do we multiply the value by 8/9. For this Taylor Riggan @ Waflhouse.com gives a nice explanation. See here -
http://waflhouse.com/?p=47
Reproducing from WAFL House website
The second part of this can begin to get a little confusing. It has to do with error-correction codes and their use on enterprise-class disk arrays such as NetApp. NetApp uses two types of disk drives – SAS disks and SATA disks. The manufacturers of these disks typically create SAS disks for business use and SATA disks for both business and home user (consumer) use. Given that fact, they include some additional functionality on SAS disks to allow enterprise-class disk subsystems to consistently validate the consistency of data on disks. They do this through something called a checksum (nothing more than a hash value that they can reference to validate data consistency). That checksum, on SAS disks, is stored in the same sector as data (over simplifying here, but this is close). The sector size on SAS disks is large enough to incorporate this (520 bytes per sector) and, as such, you really don’t see a reduction in capacity on the drives after doing the base-10 to base-2 sizing conversion. SATA disks is where it gets a bit fuzzy. Home users don’t necessarily need checksums. So, disk manufacturers don’t include that extra space within the sectors to accomodate checksums (512 bytes per sector). Well, what do enterprise-class storage vendors do in order to use checksums on SATA disks? They use up an extra 9th sector for every 8 sectors used! This eats away SATA usable capacity much faster than SAS capacity. When using SATA disks, you have to take the size you get from the base-10 to base-2 sizing conversion and multiply that by 8/9. Back to that 1TB SATA disk that we were using as an example, after base-10 to base-2 conversion we were left with 931.32GB. After checksum allocation, we’re left with 827.8GB of usable capacity
So a ITB hard disk leaves us with 827.8 GB of usable capacity so far.
STEP3: RAID GROUP DETERMINATION
Again I am not going to dwell on this much and going to give reference to cfheoh article @
http://storagegaga.wordpress.com/2011/10/20/playing-with-netapp-final-usable-capacity/
Reproducing From Storagegaga.com
RAID group is important because it is used to balance a few considerations
•Performance in recovery if there is a disk reconstruction or resilvering
•Combined RAID performance and availability through a Mean Time Between Data Loss (MTBDL) formula
Different ONTAP versions (and also different disk types) have different number of HDDs to constitute a RAID group. For ONTAP 8.0.1, the table below are its recommendation.
So, given a large pool of HDDs, the NetApp storage administrator has to figure out the best layout and the optimal number of HDDs to get to the capacity he/she wants. And there is also a best practice to set aside 2 HDDs for a RAID-DP configuration with every 30 or so HDDs. Also, it is best practice to take the default recommended RAID group size most of the time.
I would presume that this is all getting very confusing, so let me show that with an example. Let’s use the common 2TB SATA HDD and let’s assume the customer has just bought a 100 HDDs FAS6000. From the table above, the default (and recommended) RAID group size is 14. The customer wants to have maximum usable capacity as well. In a step-by-step guide,
1.Consider the hot sparing best practice. The customer wants to ensure that there will always be enough spares, so using the rule-of-thumb of 2 HDDs per 30 HDDs, 6 disks are set aside as hot spares. That leaves 94 HDDs from the initial 100 HDDs.
2.There is a root volume, rootvol, and it is recommended to put this into an aggregate of its own so that it gets maximum performance and availability. To standardize, the storage administrator configures 3 HDDs as 1 RAID group to create the rootvol aggregate, aggr0. Even though the total capacity used by the rootvol is just a few hundred GBs, it is not recommended to place data into rootvol. Of course, this situation cannot be avoided in most of the FAS2000 series, where a smaller HDDs count are sold and implemented. With 3 HDDs used up as rootvol, the customer now has 91 HDDs.
3.With 91 HDDs, and using the default RAID group size of 14, for the next aggregate of aggr1, the storage administrator can configure 6 x full RAID group of 14 HDDs (6 x 14 = 84) and 1 x partial RAID group of 7. (91/14 = 6 remainder 7). And 84 + 7 = 91 HDDs.
4.RAID-DP requires 2 disks per RAID group to be used as parity disks. Since there are a total of 7 RAID groups from the 91 HDDs, 14 HDDs are parity disks, leaving 77 HDDs as data disks.
This is where the rightsized capacity comes back into play again. 77 x 2TB HDDs is really 77 x 1.69TB = 130.13TB from an initial of 100 x 2TB = 200TB.
If you intend to create more aggregates (in our example here, we have only 2 aggregates – aggr0 and aggr1), there will be more consideration for RAID group sizing and parity disks, further reducing the usable capacity.
STEP4: SPARE DISKS
Deduct how many spares you plan to allocate.
Below is recommended from Netapp
On NetApp storage, disk failures automatically trigger parity reconstructions of affected data onto a hot standby (spare) disk, assuming that a spare disk is available. If no spare disks are available, self-healing operations are not possible. The system will run in degraded mode (requests for data on the failed disk are satisfied by reconstructing the data using parity information) until a spare is provided or the failed disk is replaced. During this time, your data is at greater risk should an additional failure occur. (With NetApp RAID-DP™, a RAID group operating in degraded mode can undergo one additional disk failure without suffering data loss.)
The number of spares you need varies based on the number of disk drives attached to your storage system. For a lower-end FAS200 or FAS2000 with a single shelf, one spare disk may suffice (configure two if you want to use Maintenance Center). On the FAS6080, with a maximum spindle count of 1,176 disks, more spare disks are needed to ensure maximum storage resiliency, especially with larger SATA disks that have longer reconstruction times.
NetApp recommends using two spares per disk type for up to 100 disk drives, where disk type is determined by a unique interface type (FC, SATA, or SAS), capacity, and rotational speed. For instance, if you have a system with 28 300GB 15K FC disks and 28 144GB 15K FC disks, you should provide four spares: two of the 300GB capacity and two of the 144GB capacity.
STEP5: Root Volume Consideration
Root volume contains all configurations files (ONTAP Files). It is always preferred to separate the root volume from the data volumes on the filer. For optimal management, it is better to restrict the root volume to just 2 disks (minimum required for a RAID-4 parity group). This enables faster reconstruction times in case of a disk failure on the root volume. Deduct how many disks you have allocated for the root volume alone.
STEP6: Volume Snapshot Reserve
Default snapshot space reserve is 20% of the volume size.
This variable is set at the volume level and is set as a percentage of the volume. Data ONTAP removes the defined percentage of volume from being available for configuring LUNs or for file usage with CIFS or NFS. As Snapshot copies need space, they consume space in the snap reserve area. By default after the snap reserve area is filled, the Snapshot copies start to take space from the general volume.
STEP7: Aggregate Reserve
Aggregates are created by default with 5% reserve space. Some people leave this to 0 and set the reserve on the volume level.
STEP8: WAFL Overhead
WAFL reserves approximately 10% of space for performance reasons, and for some part of the metadata. This is available in the new version 1.1
(A ton of thanks to my friend Guru for his help and support in the development of this tool)
Murugappan
