What does the Unified VSC WebCLI 'container rebalance' do?

Update: This post was written before I got my hands on ONTAP Tools for VMware vSphere 9.8.

Rebalancing vVols datastores

The 9.8 release of ONTAP tools supports a command to rebalance FlexVol volumes in your datacenter. The main goal is to enable even space utilization among FlexVol volumes. ONTAP tools redistributes vVols among existing volumes based on space usage, thin provisioning, LUN count, and storage capability profiles.

The rebalancing of vVols datastore is performed by LUN move or file move.



When you're constructing your VMware VVOL Datastore on NetApp ONTAP, amongst the first considerations are:

Q: How many flexvols in our VVOL Datastore?
Q: What size should we make these flexvols?

I'm going to focus on a VVOLs on FC SAN implementation. And we've got a NetApp AFF HA-pair, each controller with one aggregate.

One thing to remember with SAN is that there is a limit to the maximum number of LUNs you can have in a volume, and the documentation - Working with VVOLs (netapp.com) - mentions:
Note: It is a good practice to include multiple FlexVols in a VVOL datastore. Because FlexVols have LUN count restrictions that limit the number of virtual machines, having multiple FlexVols can increase performance.

You can actually find the 'Maximum number of LUNs in a Volume' under the 'SAN Cluster Limits' and scope 'Volume' section of NetApp's Hardware Universe (hwu.netapp.com). For instance, for an AFF A800 running ONTAP 9.7P12, this number is 1024.

So we need to think “how many VVOL VM's are we likely going to create?” For best performance you might aim for no more than 100 LUNs in a FlexVol or maximum 50% of the LUNs per FlexVol limit. And remember that each VVOL VM has:

  • One LUN for VM files
  • One LUN for swap (when the VM is powered on)
  • One LUN for every hard disk configured on the VM.
  • One LUN for memory if you take a snapshot of a powered on VM and tick the box to 'Include Virtual Machine's Memory'

Note: You can of course add more FlexVols to the VVOL Datastore later.

The point of this post though is just to consider what the Unified VSC WebCLI command 'container rebalance' does, and all it does is rebalance all the constituent FlexVols in the VVOL datastore to have the same free space. It does not move LUNs from a nearly full flexvol to a less full one (which is what I was hoping for.) So to avoid situations where an aggregate is going to run out of space because the non-space reserved LUNs have started to blow up, you'd need to consider having multiple constituent FlexVols on the aggregate so you could Vol Move one of the constituent FlexVols if you were in a tight situation.

Note: This section of TR-4400 suggest SCP / VM Storage Policy might be a way to move VVOL LUNs:
VASA Provider 7.1 can also automatically migrate a VVol if the datastore contains multiple FlexVol volumes, and a different volume supports the changed policy. For example, if a VVol datastore contains encrypted and unencrypted FlexVol volumes and policy is changed for encryption, ONTAP moves the VVol as appropriate.”

Container Rebalance Example

Note: I did document the VSC WebCLI core commands here:
Cosonok's IT Blog: NetApp VSC, VASA Provider, and SRA virtual appliance for ONTAP - Control Panel

Before running 'container rebalance':

cluster1::> vol show *Vvol*
Vserver Volume Size Available Used%
--------- -------------- -------- --------- -----
svm1 iscsi_Vvol_fv1 20GB 17.19GB 14%
svm1 iscsi_Vvol_fv2 20GB 17.20GB 14%
svm1 iscsi_Vvol_fv3 20GB 18.61GB 6%
svm1 iscsi_Vvol_fv4 25GB 25.00GB 0%

Then we run (see image below):
container rebalance -container_name=iscsi_vVol_test1

And the result is simply that the free space in all these flexvols ends up as being identical.

cluster1::> vol show *Vvol*
Vserver Volume Size Available Used%
--------- -------------- -------- --------- -----
svm1 iscsi_Vvol_fv1 22.30GB 19.50GB 12%
svm1 iscsi_Vvol_fv2 22.30GB 19.50GB 12%
svm1 iscsi_Vvol_fv3 20.89GB 19.50GB 6%
svm1 iscsi_Vvol_fv4 19.50GB 19.50GB 0%

Image: Running 'Container Rebalance'