Saturday, 27 December 2014

More on Tracking Quotas, with Dedupe and Volume Autosize Considerations

1) A correction to this post from July 2014: Volume and Qtrees Quotas in Clustered ONTAP: Tracking Quota

To enable tracking quotas for all Qtrees in a volume (you do not need to create a tracking rule for every qtree), it is simply ::>

volume quota policy rule create -vserver SVMNAME -policy-name POLICYNAME -volume VOLUMENAME -type tree -target "" -threshold "-"
volume quota on -vserver SVMNAME -volume VOLUMENAME

{Where POLICYNAME is the currently active Quota Policy on the SVM, with default policy being default}

When the volume quota has finished initializing (quota show -vserver SVMNAME -volume VOLUMENAME), to get a report for ‘Disk Used’ and ‘Files Used’ for all Qtrees in that volume ::>

volume quota report -vserver SVMNAME -volume VOLUMENAME

2) Quotas and Volume Autosize Considerations

From TR 3505 NetApp Deduplication:

6.4.1 QUOTAS
For deduplicated files, the logical (undeduplicated) size is charged against the quotas. There are several advantages to this scheme as opposed to charging quotas based on the physical (deduplicated) size of the file:
i) This is in line with the general design principle of making deduplication transparent to the end user.
ii) It is easier for system administrators to manage quotas. They can maintain a single quota policy across all volumes, whether or not deduplication is enabled on it.
iii) There are no out-of-space failures when data is being sent using SnapMirror from a volume with deduplication enabled to a destination volume that has deduplication disabled.

To understand the effect of seeing the logical (undeduplicated) size when using Qtree quotas, here are 3 examples. In each example, we have a 1000MB thin-provisioned volume (space guarantee none), with no snapshots, dedupe is enabled on the volume, and Qtree tracking quotas are enabled. We use the same 210 MB folder multiple times (so it de-dupes very well!) The Storage Virtual Machine and AD Machine Account is called cifsv1.

2.1) Example 1: 4 x the same 210MB folder (~840MB) in one Qtree

Image: Example 1 Data Layout
DOS Commands to Map Drives:

net use V: \\cifsv1\vol1
net use Q: \\cifsv1\vol1\qtree1
net use R: \\cifsv1\vol1\qtree2

Image: Displayed via a Windows Client for Example 1
2.2) Example 2: The same 210MB folder (~840MB) in 3 Qtrees and the root of the Volume

Image: Example 2 Data Layout
DOS Commands to Map Drives:

net use V: \\cifsv1\vol2
net use Q: \\cifsv1\vol2\qtree2

Image: Displayed via a Windows Client for Example 2
2.3) Example 3: 5 x the same 210MB Folder (~1050MB) in one Qtree

Image: Example 3 Data Layout
DOS Commands to Map Drives:

net use V: \\cifsv1\vol3
net use Q: \\cifsv1\vol3\qtree3

Image: Displayed via a Windows Client for Example 3
3) Volume Autosize Considerations

Generally, when dedupe and Qtree quotas are in effect, the free space the end user will see is:

End User Free Space = {Volume Size} - {Logical (Un-deduped) Data Size in the Qtree}

- or, if the actual (including dedupe) data size in the volume is larger than the logical data size in the Qtree -

End User Free Space = {Volume Size} - {Actual (Including Dedupe) Data Size in the volume}

We see in example 1 above, the volume share shows the actual free space available in the volume as 797 MB, but the qtree share shows a much smaller free space value of 150 MB. This has implications regards setting volume autosize, in essence:

Volume AutoSize Size Setting = ( {Volume Size} - {Logical (Un-deduped) Data Size in the Largest Qtree} + {Actual (Deduped) Data Size in the Volume} )* {Percentage Autogrow Threshold}

- or, if the actual (including dedupe) data size in the Volume is larger than the logical data size in the largest Qtree, we get the typical formula -

Volume AutoSize Size Setting = {Volume Size} * {Percentage Autogrow Threshold}

In example 1, even though the volume is only 203 MB used out of 1000 MB, if the user is accessing Qtree1, and tries to add more than 150 MB, this will fail (even though there is 797 MB free space in the volume), and the autosize will not have triggered if it were set at 850MB.

Example 1: AutoSize Size Setting = ( 1000 - 850 + 203 ) * 0.85 ~= 300 MB
Example 2: AutoSize Size Setting = 1000 * 0.85 = 850 MB

Note: Interestingly, in Example 3, you might expect to see negative user space when we have 1050 MB logical data in a Qtree in a 1000 MB volume, but that doesn’t happen, the Windows client then shows the deduped data size in the volume!

4) Final Comment

If your end-user is saying his share is full and he needs more space, but the volume is far from being full, nor is the aggregate, and volume autosize has not kicked in as expected, suspect dedupe and quotas!

1 comment:

  1. Thanks for the post we faced similar issue , your explanation makes sense

    ReplyDelete