One of the most misunderstood topics with NetApp FAS systems is reallocation. There are two types of reallocation that can be run on these systems; one for files and volumes and another for aggregates. The process is run in the background, and although the goal is to optimize placement of data blocks both serve a different purpose. Below is a picture of a 4 disk aggregate with 2 volumes, one orange and one yellow.
If we add a new disk to this aggregate, and we don’t run volume level reallocation all new writes will happen on the area in the aggregate that has the most contiguous free space. As we can see from the picture below this area is the new disk. Since new data is usually the most accessed data you now have this single disk servicing most of your reads and writes. This will create a “hot disk”, and performance issues.
Now if we run a volume reallocation on the yellow volume the data will be spread out across all the disks in the aggregate. The orange volume is still unoptimized and will suffer from the hot disk syndrome until we run a reallocation on it as well.
This is why when adding only a few new disk to an aggregate you must run a volume reallocation against every volume in your aggregate. If you are adding multiple disks to an aggregate (16, 32, etc) it may not be necessary to run the reallocate. Imagine you add 32 disk to a 16 disk aggregate. New writes will go to 32 disk instead of the 16 you had prior so performance will be much better without taking any intervention. As the new disk begin to fill up writes will eventually hit all 48 disks in your aggregate. You could of course speed this process up by running manual reallocation against all volumes in the aggregate.
The other big area of confusion is what an aggregate reallocation actually does. Aggregate reallocation “reallocate -A” will only optimize free space in the aggregate. This will help your system with writes as the easier it is to find contiguous free space the more efficient those operations will be. Take the diagram below as an example of an aggregate that could benefit from reallocation.
This is our expanded aggregate that we only reallocated the yellow volume. We see free space in the aggregate where the blocks were distributed across the other disk. We also see how new writes for the orange volume stacked up on the new disk as that is where we had the most contiguous free space. I wonder if the application owner has been complaining about performance issues with his orange data? The picture below shows us what happens after the aggregate reallocate.
We still have the unoptimized data from the volume we did not reallocate. The only thing the aggregate reallocate did was make the free space in it more contiguous for writing new data. It is easy to see how one could be confused by these similar but different processes, and I hope this helps explain how and why you would use the different types of reallocation.
Credits go to Erick Moore of http://www.theselights.com .