Saturday, 31 August 2013

Clustered ONTAP 8.2 SIM: Maximizing Available Usable Space

The NetApp Clustered ONTAP (CDOT) 8.2 Simulator - as first downloaded - has two virtual shelves of 14 x 1GB disks. With 3 disks taken for the dedicated Clustered ONTAP root aggregate, which leaves 25 for a data aggregate, of which two of those disks will be parity disks, giving a maximum usable space of around 23GB. If you want to see how it is possible to get close to 400GB usable space out of your CDOT 8.2 Simulator, read on!

Walkthrough - how to get the most out of your CDOT 8.2 SIM, starting from the very start!

Part 1: Obtaining the SIM

1: Download the CDOT 8.2 SIM from > Downloads > Product Evaluations > Data ONTAP Simulator > Simulator 8.x > 8.2 Clustered-ONTAP for VMware Workstation
2: Unpack the downloaded vsim_netapp-cm.tgz
3: Open the DataONTAP.vmx file with VMware Workstation

Part 2: First Boot

1: Boot the simulator
2: Press Ctrl-C for Boot Menu when prompted
3: Enter selection 4 ‘Clean configuration and initialize all disks’ and answer ‘y’ to the two prompts

Part 3: Replacing the 2x14x1 GB Virtual Disks with 4x14x9 GB!

1: After the wipe request from part 2, the SIM will load to ‘Do you want to create a new cluster ...’ press Ctrl-C to escape the setup script
2: The login is ‘admin’ with no password
3: Run through the following commands/prompts>

security login unlock -username diag
security login password -username diag
Please enter a new password: XXXXXXXX
Please enter it again: XXXXXXXX
set -privilege advanced
Do you want to continue? y
systemshell local
login: diag
Password: XXXXXXXX
setenv PATH "${PATH}:/usr/sbin"
echo $PATH
cd /sim/dev/,disks
sudo rm v0*
sudo rm v1*
sudo rm ,reservations
cd /sim/dev
vsim_makedisks -h
sudo vsim_makedisks -n 14 -t 36 -a 0
sudo vsim_makedisks -n 14 -t 36 -a 1
sudo vsim_makedisks -n 14 -t 36 -a 2
sudo vsim_makedisks -n 14 -t 36 -a 3
ls ,disks/
system node halt local
Warning: Are you sure you want to halt the node? y

Note: The SIM comes with 2 virtual shelves of 14 x Type 23 (1GB) disks. Here we’re simply trashing those 2 shelves and adding 4 shelves (which is as many as you can add) of 14 x Type 36 (~9GB) disks.

Part 4: Second Wipe

1: Power off the SIM
2: Edit the properties of the Virtual Machine to make Hard Disk 4, 550 GB in size

Image: Expanding VMware Workstation VMDK
3: Power on the SIM
4: Press Ctrl-C for Boot Menu when prompted
5: Enter selection 5 ‘Maintenance mode boot’
6: Assign 3 disks for the Clustered ONTAP dedicated root aggregate, and halt

disk assign v4.16 v4.17 v4.18
disk show

7: Power-cycle the SIM
8: Press Ctrl-C for Boot Menu when prompted
9: Enter selection 4 ‘Clean configuration and initialize all disks’ and answer ‘y’ to the two prompts

Part 5: Cluster Setup and Aggregate Setup

1: After the wipe request from part 2, the SIM will load to ‘Do you want to create a new cluster ...’ follow the prompts to create your cluster

Note: SIM Base License Key is SMKQROWJNQYQSDAAAAAAAAAAAAAA (that’s 14 A’s!)

2: Login to the cluster
3: Run through the following commands/prompts to create the largest CDOT 8.2 SIM aggregate possible!

storage disk assign -all true -node lab-01
system node run -node lab-01 options disk.maint_center.spares_check off
storage aggregate create -aggregate a_lab01_01 -diskcount 53 -nodes lab-01 -maxraidsize 28

And the output to show the nearly 400GB lab simulator aggregate!

lab::> storage aggregate show
  (storage aggregate show)
Aggregate     Size Available Used% State
--------- -------- --------- ----- ------
a_lab01_01 387.6GB   387.6GB    0% online
aggr0       7.91GB   379.0MB   95% online


  1. nice job. I couldn't find the makedisks.main from the previous v8.1x. My "ls -R" turned up vsim_makedisks but I couldn't find the path to the location. I have no idea how you figured it all out, but apparently you did so Kudos to you.

  2. Thank you so much for posting this Vidad! You've made my home lab much more usable as I can now dedupe :)

    1. No problem Phil. Thanks for reading :-)

  3. Hi, I have setup CDOT Simulator, however I have issue creating Volume, it throws this error, could you please share some light on this?
    cdot01::volume> volume create -vserver cdot01-01 -volume oradata1 -aggregate aggr0 -size 1g -state online

    Error: command failed: This operation is not allowed on a node Vserver.

    1. Hi Manju,
      I'd imagine you've worked this out now. But, yes, you do not create volumes on the Node Vserver, you create volumes on a Cluster (Data) Vserver.
      PS I should really be saying SVM (Storage Virtual Machine) instead of Vserver, but I think Vserver rolls off the tongue easier.

  4. Thanks for sharing.

    what if I wanted to add 2nd node?
    I'd follow the same step, but answer 'no' on the question of ‘Do you want to create a new cluster ...’, since I want to join an existing cluster.


    Looking forward your comments.

    1. Pretty much, you'll be asked:

      Do you want to create a new cluster or join an existing cluster?

      And for the second node you type "join"

    2. Here is my problem:
      After I enter selection 4 ‘Clean configuration and initialize all disks’ and answer ‘y’ to the two prompts in PART2, , the SIM did NOT load to ‘Do you want to create a new cluster ...’ in PART3. So, I don't have the chance to press ctl-c and to escape the setup script.

      I had already done with setting up the 2-node cluster with the default size. So, I reboot the first node, and then CTRL-C to bring up boot menu, and then choose options 4.. then following your instruction.

      Do I have to start over from scratch by running VMware-player-6.0.3-1895310.exe?

      Thanks a lot for your help!

    3. These instructions are for pre-cluster create/join. If you already had a 2-node cluster in VMware player, don't see why you can't recreate this as a new 2-node cluster (the serial numbers/sysid's must already have been set different for the join to work.) If you're trying to add a modified node to the old cluster, you might have challenges.

    4. Well, I have an issue when creating cluster2. I know it is beyond the topic here, but I post the issue on netapp community, could not get any solution yet.

      In the end, cluster2 is created, but cluster mgmt. IP is not pingable from cmd prompt. I can ping node mgmt. ip, and I can ping cluster mgmt. ip inside cluster2.

      I tried quite some different things. I used the same subnet as cluster1, of cause, different number in the end.

      Could you please offer any idea here?

  5. Can confirm it works on simulator 9.0 RC1...
    slight changes.

    FAILED TO RUN - security login unlock -username diag.
    you only need to run "security login password -username diag" not the unlock.

    FAILED TO RUN - Systemshell local
    when it asks you to write "systemshell local" it failed so I just typed exit and it dropped back to a login, login with diag as normal.

    I also did this on esxi 5.5 cluster and had to manually extend the VDKM on the esxi cli using vmfstools -X 550g _1.vmdk once that was complete a then converted to thick from the esxi datastore file veiwer by right clicking and select inflate. Not sure if that's required but did it anyway (took awhile)