Another Blog Post on NetApp Volume Reallocate

There I was thinking this would make a great post, only to discover the use of reallocate has been pretty much blogged to death by now (see a brief listing in the Appendix below). Not to be perturbed, here’s my 2 cents!

Here we’re only considering:
The use of volume reallocate after adding disks to an aggregate

Q1: What is volume reallocate?

In a nutshell - volume reallocate spreads the blocks of a volume “evenly” across all the spindles in the aggregate, or at least that’s the idea.

Q2: What are the best practices of using reallocate?

Please refer to:

Reallocate Best Practices TR-3929

And in section 2.5 there’s the:

Best Practice: Use forced reallocation after adding disks to an aggregate so that existing data is spread onto the new disks, reducing the time required to balance the load across all the disks.

Q3: What’s the right practice?

Okay, the above might be the “best practice” but there are few things to consider here when we think about “right practice”:

1) How many disks are being added and is this likely to cause a “hotspot”?
2) How utilized are the controllers and existing disks?
3) How many volumes in the aggregate and what sizes?
4) How long is the reallocate going to take?
5) What applications are running from these volumes?
6) Using deduplication?
7) Is any performance impact of adding the disks noticeable/can it wait?
8) Any volume that is a snapmirror target will need to resync after the reallocate is complete

On 1) For example – if adding 2 disks to a 14 disk RAID group to make a 16 disk RAID group, those 2 disks will be a hotspot. If adding 2 x 14 disk RAID groups to an existing 14 disk RAID group, they’ll be no hotspot caused by adding the new disks.

On 2) The volume reallocate adds load to the system. If the controllers are already close to CPU capacity, and the existing aggregate disks are highly utilized, the volume reallocate may need to be held off until a maintenance weekend, or period of low utilization.

On 3) The reallocate is done one volume at a time. If there are a lot of small volumes in the aggregate, the potential performance impact of volume reallocation to end users is going to be less than if there are fewer larger volumes.

On 4) The reallocate is done with the smallest volumes first. After a few of the smaller volumes have been run, it might become apparent that some of the larger volumes will be running well into peak times which may not be desirable depending on system performance (reallocate can always be terminated if it is overrunning.)

On 5) It is worth considering the applications running off these volumes - are they intolerant to latency. How is running the reallocate going to impact the performance? Or, is the performance benefit of having a reallocated volume across all spindles more desirable?

On 6) Check with NGS (NetApp Global Support) before running reallocate if they have deduplication enabled, or look for whether any BURTs related to reallocate on deduped volumes might affect the customers controller and version of Data ONTAP.

On 7) Essentially, this task is a case of do one volume at a time, starting with the smallest, and monitor what/if any performance impact the reallocate causes (weighing this against the advantages of having fully “reallocated” volumes). On the other hand, if there are no concerns with system performance after the disks have been added to the aggregate, they could just leave it and it will sort itself out in time J

Q4: What command do we run?

FAS> reallocate start -f -p /vol/volname

(-f) “Forced reallocation ignores the optimization thresholds and completely rewrites the data to disk, unlike the normal reallocation process. Although this improves the layout, routine use of reallocate -f is not a best practice.”
(-p) “Because all of the data is optimized, forced reallocation cannot be run against volumes that have existing Snapshot copies unless the physical reallocation method (-p) is also used.”

Q5: What other options are there for the reallocate command?
Q6: And why not aggregate reallocation?

FAS> reallocate


reallocate on | off
reallocate start [-t threshold] [-p] [-o] [-n] [-i interval] | /vol/
reallocate start -f [-p] | /vol/
reallocate start -A [-o] [-i interval]

NOTE: -A is for aggregate (freespace) reallocation. Do NOT use -A after growing an aggregate if you wish to optimize the layout of existing data; instead use reallocate start -f /vol/ for each volume in the aggregate.

reallocate status [-v] [ | ]
reallocate stop |
reallocate quiesce |
reallocate restart [-i] |
reallocate schedule {-d | -s } |
reallocate measure [-l logfile] [-t threshold] [-o] [-i interval] | /vol/

APPENDIX: Much Better Posts on Reallocate Than This One

NetApp Reallocate - What is Reallocation?

NetApp Aggregate Expansion

Understanding NetApp Volume and Aggregate Reallocation

Image: Nice graphic (more in the post)

My first experience adding disks and reallocation

Image: Nice chart with the effects before shelf, before reallocate, and after reallocate

And there are many more too - sorry if I’ve missed yours out!