Monday, 20 March 2017

NetApp CN1610 Cluster Switch Notes: How to Setup

I realized I’ve done loads of CN1610s but never blogged my notes (always referring to notes on my laptop), so here we go (cabling is not covered here, just switch setup.)

Image: NetApp CN1610 Cluster Switches with ISLs Cabled

1) Initial Cluster Switch Setup Script

Connect a laptop to the switch’s console (RJ45) port. Out-of-the-factory, the default CN1610 login is:

username = admin
password = {no password}

You will initially be in the user USER command mode: (CN1610) >
From here, copy and paste the below script, with the highlighted entries updated accordingly:

serviceport protocol none
network protocol none
hostname SWITCH_NAME
username admin password PASSWORD
write memory

Repeat on both switches. To verify serviceport and network settings:

(CN1610) # show serviceport
(CN1610) # show network

Note: Type ‘en’ or ‘enable’ to get from the USER command mode - (CN1610) > - to the EXEC mode - (CN1610) #

2) Cluster Switch OS and RCF File

At the time of writing, the recommended versions of Fastpath OS and RCF File for CN1610 cluster switches with ONTAP 8.3 to 9.1 are:

RCF Version = 1.2

The Compatibility Matrix is at the link below (software and instructions are available off this link too):

To check FASTPATH and RCF (usually they’re correct out-of-the-factory), run the below commands - the RCF version is listed in the description for interface 3/64:

(CN1610) # show version
(CN1610) # show running-config

Upgrading FASTPATH and/or the RCF requires a TFTP server.

If you need to upgrade the RCF (NetApp recommends doing the RCF before OS), the commands are:

(CN1610) # show running-config config_backup.scr
(CN1610) # copy tftp://tftpserver/CN1610_CS_RCF_v1.2.txt nvram:script CN1610_CS_RCF_v1.2.scr
(CN1610) # script list
(CN1610) # script apply CN1610_CS_RCF_v1.2.scr
(CN1610) # show running-config
(CN1610) # write memory
(CN1610) # reload

If you need to upgrade FASTPATH, the commands are:

(CN1610) # show bootvar
(CN1610) # copy active backup
(CN1610) # show bootvar
(CN1610) # copy tftp://tftp_server_address/NetApp_CN1610_1.2.0.7.stk active
(CN1610) # reload

(CN1610) > enable
(CN1610) # show version

3) Configuring DNS, NTP, and SSH

Configure DNS, NTP and SSH using the commands below, with the highlighted entries updated accordingly:

(CN1610) # config
(CN1610) (Config)# ip domain name DNS_DOMAIN_NAME
(CN1610) (Config)# ip name server NAME_SERVER_1,NAME_SERVER_2,...
(CN1610) (Config)# sntp client mode unicast
(CN1610) (Config)# sntp server NTP_SERVER_1
(CN1610) (Config)# sntp server NTP_SERVER_2
(CN1610) (Config)# clock timezone +/- NUMBER*
(CN1610) (Config)# exit
(CN1610) # show ip name server
(CN1610) # show sntp

(CN1610) # ip ssh protocol 2
(CN1610) # config
(CN1610) (Config)# crypto key gen rsa
(CN1610) (Config)# crypto key gen dsa
(CN1610) (Config)# exit
(CN1610) # ip ssh server enable
(CN1610) # show ip ssh

*UTC 0 is 0, UTC -1 is -1, UTC +2 is 2, etcetera.
Note: SSH needs to be enabled before running Config Advisor. The RCF version 1.2 disables telnet (for security reasons) with the line "ip telnet server enable".

4) Passwords

To change the current logged in user’s password:

(CN1610) > password

To set an enable password:

(CN1610) # enable password PASSWORD

5) Configuring Switches for Email Alerting

Use the following commands starting from the EXEC privilege level (#). Replacing highlighted entries as required:

mail-server MAIL_SERVER
logging email
logging email 3
logging email urgent 2
logging email message-type both to-addr MAIL_TO_ADDRESS
logging email from-addr MAIL_FROM_ADDRESS
logging email message-type both subject "Alert from SWITCH_NAME"
logging email test message-type both message-body "THIS IS A TEST"
show logging email config
show running-config
write memory

To test email alerting from the EXEC privilege level (#):

logging email test message-type both message-body "THIS IS A TEST - PLEASE IGNORE!"

6) Saving Changes (IMPORTANT)

To save changes so that they are persistent to reboots:

(CN1610) # write memory

Saturday, 18 March 2017

Upgrade ONTAP 8.3.1 to 9.1 Walkthrough

Back in October I posted Cluster Software Update Essentials which outlined 3 different upgrade methods (really, methods 1 & 2 were the same, just GUI v CLI.) In this post I’m going to run through the method I’d use - “cluster image update” (for anything other than single node clusters.) Upgrading from 8.3.0 to 9.1 is a one step process.


1) Verify your environment is supported for ONTAP 9.1
- and make remediation’s where required.
Note: This includes FC SAN hosts, boot from SAN hosts, OFFTAP products…
2) Verify platform support, and root volume sizes
3) Verify cluster health via AutoSupport
4) Verify cluster configuration with Config Advisor
5) Verify cluster health in OnCommand Unified Manager.
6) Thoroughly go over the Upgrade Advisor outputs and do the appropriate!
7) Download the ONTAP 9.1 image
8) Serve the 91P1_q_image.tgz file on a web server.

Note: 9.1P1 is the recommend release at the time of writing.

Cluster Image Package Get

Use the highlighted commands below to get the new image on the cluster, and perform validation:

cluster1::> cluster image show
Node        Current Version Installation Date
----------- --------------- -------------------
cluster1-01 8.3.1           9/17/2015 16:54:16
cluster1-02 8.3.1           9/17/2015 16:55:04

cluster1::> cluster image package show-repository
There are no packages in the repository.

cluster1::> cluster image package get -url

Software get started on node cluster1-01
Downloading package. This may take up to 10 minutes...

cluster1::> cluster image package show-repository
Package Version Package Build Time
--------------- ------------------
9.1P1           2/14/2017 13:14:46

cluster1::> cluster image validate -version 9.1P1

It can take several minutes to complete validation...

Pre-update Check                  Status
----------------------------      ------
Aggregate plex resync status      OK
Aggregate status                  OK
Autoboot Status                   OK
Broadcast Domain status           OK
CIFS status                       OK
CPU Utilization Status            OK
Cluster health status             OK
Cluster quorum status             OK
Data ONTAP Version Status         OK
Disk status                       OK
High Availability status          OK
Jobs Status                       OK
LIF failover                      OK
LIF load balancing                OK
LIFs not hosted                   OK
LIFs on home node status          OK
Manual checks                     Warning: Manual validation checks need to be performed. Refer to the Upgrade Advisor...
MetroCluster configuration status OK
NDMP status                       OK
NFS netgroup check                OK
Platform status                   OK
Previous Upgrade Status           OK
SAN LIF status                    OK
SAN status                        OK
Security Config SSLv3 check       OK
SnapMirror status                 OK
Snapshot copy count check         OK
Volume move status                OK
Volume status                     OK
Overall Status                    Warning

Update the first HA Pair

Note the following warning from Upgrade Advisor regards SnapMirror:
“To prevent SnapMirror transfers from failing, you must suspend (Quiesce) SnapMirror operations and upgrade destination nodes before upgrading source nodes.
(i) Suspend SnapMirror transfers for a destination volume
(ii) Upgrade the node that contains the destination volume
(iii) Upgrade the node that contains the source volume
(iv) Resume the SnapMirror transfers for the destination volume”

This is a good point at which to send AutoSupports (and verify they have sent)::>

autosupport invoke -node * -type all -message "maint=4h upgrading to ONTAP 9.1P1"
autosupport history show -node cluster1-01
autosupport history show -node cluster1-02

Note: If this is a 4-node or greater cluster, you might like to make sure Epsilon is not on the HA-Pair being upgraded (cluster image update will handle this.)

set adv
cluster show
cluster modify -node EPSILON_NODE -epsilon false
cluster modify -node NEW_EPSILON_NODE -epsilon true

Update the HA-Pair:

cluster1::> cluster image update -version 9.1P1 -nodes cluster1-02,cluster1-01

Starting validation for this update. Please wait...
It can take several minutes to complete validation...
Warning: Validation has reported warnings. Do you want to continue? {y|n}: y
Starting update...

Note: You don’t need to do 1 HA-pair at a time, “cluster image update” is designed to update the entire cluster.

Verifying the Update Process

You should be connected to Service-Processors to observe the update process.
Note: If you have aggregates serving CIFS, you may need to do a “storage failover giveback -ofnode NODE -override-vetoes true”

cluster image show-update-progress
storage failover show-takeover
storage failover show
storage failover show-giveback

Update Subsequent HA Pairs

Follow the above to update subsequent HA Pairs as required.


Verify the cluster version has increased (the cluster version only updates when all nodes are upgraded).
If you suspended SnapMirrors re-enable them.
Send AutoSupports::>

snapmirror resume -source-path {SRC_PATH} -destination-path {DEST_PATH}
autosupport invoke -node * -type all -message "maint=END finished upgrade to ONTAP 9.1P1"

And test your environment.

Appendix: (Some) Supported things with ONTAP 9.1

Just a personal reference to save some IMT results (correct at 2017.03.18):

OnCommand Unified Manager 7.0
OnCommand Unified Manager 7.1
OnCommand Performance Manager 7.0
OnCommand Performance Manager 7.1
SnapDrive 7.1.3 for Windows
SnapDrive 7.1.4 for Windows
SnapCenter Host Plug-in 1.1 for Microsoft Windows
SnapCenter Host Plug-in 1.1 for UNIX
SnapCenter Host Plug-in 2.0 for Microsoft Windows
SnapCenter Host Plug-in 2.0 for UNIX
SnapCenter Host Plug-in 2.0 for VMware vSphere
Virtual Storage Console 6.1
Virtual Storage Console 6.2
Virtual Storage Console 6.2.1
SnapDrive 5.2.2 for Unix*
SnapDrive 5.3 for Unix*
SnapDrive 5.3.1 for Unix*

SnapCenter Application Plugin has dependencies with:
- SnapCenter Host Plugin (if 1.1 use 1.1/if 2.0 use 2.0)
- SnapCenter Server
- Host Application

SnapCenter Server has dependencies with:
- SnapCenter Host Plugin (if 1.1 use 1.1/if 2.0 use 2.0)
- SnapCenter Application Plugin
- Virtual Storage Console (VMware)

SnapManager for Exchange has dependencies with:
- SnapDrive (for Windows version)
- Host OS (Windows Server version)
- Host Application (Exchange Server version)

CN1610 Cluster Switch FASTPATH and RCF 1.2
Cisco NX3132V Cluster Switch NX-OS 7.0(3)I4(1) and RCF 1.1
Cisco NX5596 Cluster Switch NX-OS 7.1(1)N1(1) and RCF 1.3
Cisco NX5020 Cluster Switch NX-OS 5.2(1)N1(8b) and RCF 1.3
Cisco NX5010 Cluster Switch NX-OS 5.2(1)N1(8b) and RCF 1.3

Sunday, 12 March 2017

Notes on ‘NCSA - Hybrid Cloud: NS0-146’

I’ve just completed the curriculum for the ‘NetApp Certified Storage Associate - Hybrid Cloud’, and thought I’d jot down a few notes, curriculum highlights, and add in some further reading/viewing. The exam is to come!

NCSA - Hybrid Cloud: Curriculum Highlights

To find the curriculum, go to and search for “NetApp Certified Storage Associate - Hybrid Cloud”. If you’re already well versed in NetApp, it’s going to be the “Cloudy” elements of the curriculum that interest you the most, and these are highlighted below:

- Storage Fundamentals (45 mins)
- Virtualization Technology Fundamentals (20 mins)
- Cloud Fundamentals (30 mins)
- Flash Fundamentals (30 mins)
- Data Network Fundamentals (15 mins)
- ONTAP NAS Fundamentals (45 mins)
- Technical Overview of AltaVault Technology (90 mins)
(especially: “Module 4: Cloud Based AltaVault Appliance”)
- Technical Overview of StorageGRID Webscale (60 mins)
- Technical Overview of OnCommand Management Solutions for Clustered Data ONTAP (90 mins)
- Technical Overview of OnCommand Insight (120 mins)
- Knowledge Bytes: NetApp Private Storage for Amazon Web Services
+ Deploying NetApp Private Storage for Amazon Web Services:
- OnCommand Cloud Manager and Cloud ONTAP Videos

Further Reading/Viewing

The curriculum was written last year (2016), and - with the rampant advance of technology - is understandably already a little out of date, check out the following YouTube link for even more videos on NetApp “Cloudy” stuff including integrations with Azure:

Other links:

Random Screen Grabs

AltaVault: Some Use Cases

Image: AltaVault Use Cases for Backup and Archive
Image: AltaVault Use Cases for Disaster Recovery
NetApp Private Storage for AWS

Image: The Solution: NetApp Private Storage for AWS
NetApp Private Storage for AWS: Some Use Cases

Image: Disaster Recovery
Image: Development and Testing
Image: Cloudburst for Peak Workloads
Image: Multiregion Application Continuity
Image: Cloud-Centered Direct Connection Consolidation

Friday, 3 March 2017

NetApp ONTAP - API Primer: Programmatical Access to ONTAP

The post specifically focuses around API/Programmatical/Script methods/resources to help you manage your NetApp (Clustered Data) ONTAP environment.


Definitely CLI and perhaps not obviously API, but there are a few Clustershell commands that are worth looking at::>

set advanced
security login role show-ontapi

set diag
system ontapi changes show
system ontapi limits show

Tip: The output of ‘security login role show-ontapi’ shows the ONTAPI and related Clustershell command.

NetApp PowerShell Toolkit for Windows

“The ONTAP PowerShell module contains 2050 cmdlets, enabling the storage administration of NetApp ONTAP systems via ZAPI” (Toolkit version 4.3)

Tip: The output of Get-NcHelp shows the API leveraged by the cmdlet.

NetApp Manageability SDK (NMSDK)

You can find this on the download site - - under ‘NetApp Manageability SDK’. Under the {Select Platform} dropdown, there is:

- .NET API Bindings for Windows
- Java API Bindings
- All Platforms

As an example from the ‘All Platforms’ download, in the zedi folder, you will find the application zexplore.exe. A screenshot of the ‘ZExplore Development Interface’ is below:

Image: ZExplore Development Interface
You can choose your API version (sample list of ONTAP version to API version is in the Appendix below), choose your API, and get code examples for these languages:
C#, C, Java, Perl, Python, Ruby
Note: The relevant libraries are in the lib folder.

OnCommand Workflow Automation REST API

You can manage your ONTAP clusters with WFA and - furthermore - use REST API with WFA. Check out the latest version of the ‘REST Web Services Primer’ for your version of WFA. Links to WFA documentation and the latest (non-release candidate) version of the Primer are below:

OnCommand API Services (REST API)

OnCommand API Services (for Linux - CentOS or RedHat) is available from the NetApp download site.

The latest version of OnCommand API Services is 1.2. Check out the documentation:

The following image illustrates how the API server works:

Image: OnCommand API Services - how the API server works
Appendix: Sample list of ONTAP version to API version

ONTAP 9.1: Ontapi 1.110 Cluster-Mode
ONTAP 9.0: Ontapi 1.100 Cluster-Mode
ONTAP 8.3.2: Ontapi 1.32 Cluster-Mode
7-Mode 8.2.3: Ontapi 1.21 7-Mode
7-Mode 8.1.4: Ontapi 1.19 7-Mode

Note: My method of finding this information wasn’t very scientific, just from the PowerShell toolkit -

(Connect-NcController CDOT_CLUSTER -credential admin).Ontapi
(Connect-NaController 7MODE_CONTROLLER -credential root).Ontapi

- using what VSIMs I had in my lab.

NetApp ONTAP 9.1: OnCommand System Manager On-Box Guided Cluster Setup

I was never a fan of ‘System Setup’ (the GUI way of setting up pre-9.1 clusters), so when I heard that ONTAP 9.1 has an On-Box Cluster Setup, I wanted to try it out, so here we go!

Note i: With the VSIM you can’t use Guided Setup to create anything other than a single node cluster, if you try for a 2 node cluster it will complain that “HA is not configured”, but it would let you part do the first node in the cluster, and then you’d need to add the second VSIM node using CLI. With real world systems you won’t encounter this.

Note ii: Another problem I found with using Guided Cluster Setup against a VSIM, is that it gets stuck at ‘Step 2: Network’. The error is fairly vague, I’m pretty sure it’s due to the absence of service-processor on the VSIM. Again, with real world systems you won’t encounter this. The moral of the story “don’t use Guided Cluster Setup to setup a VSIM.”

Here we have 1 x ONTAP 9.1 simulator. The basic setup is similar to here - Build Recipe for 8.3.2 - up to and including step 1.8 (node setup).

Image: Node management interface configuration via the console

Guided Cluster Setup

1) Point your web browser at https://NODE_MGMT_IP

And the below is what you should see. A very cool feature is that you can download/upload a template (in CSV format - see the Appendix for more).

Image: Welcome to the Guided Cluster Setup

2) Click the ‘Guided Setup’ button.

Image: Guided Setup

2.1) Cluster

- Input Cluster name
- Input Node name
- Tick the single node cluster box
- Input admin password
- Click Submit

Image: Guided Cluster Setup 1 - Cluster

Note: The Cluster Base License is now optional in 9.1’s setup (mandatory to add Feature Licenses).

2.2) Network

- Input network details as required (see image below)
- Click Submit

Image: Guided Cluster Setup 2 - Network

... and this is where we get stuck with the VSIM. Still, hopefully this post wasn’t a total fail and you can see the potential (check out the Appendix for the CSV details!)

Image: Error in the input fields. Verify the field values before proceeding.

Appendix: Cluster Setup Template

The Cluster Setup Template functionality will be very useful. The CSV/XLSX can be downloaded from ‘Guided Cluster Setup’. I’m not going to copy and paste the CSV template here, just detail the fields to give an idea of what it could be used for (up to 8 node cluster setup!)

Section: Cluster
Cluster Name
Configuration Type(for SingleNodeCluster)
Configuration Type(for TwoNodeCluster)
Base License
Feature License(Optional)

Section: Network
Network Configuration Type
IP Address Range1
IP Address Range2
IP Address Range3
Cluster MgmtDetails(Manual)
Node-2 MgmtDetails(Manual)
Node-3 MgmtDetails(Manual)
Node-4 MgmtDetails(Manual)
Node-5 MgmtDetails(Manual)
Node-6 MgmtDetails(Manual)
Node-7 MgmtDetails(Manual)
Node-8 MgmtDetails(Manual)
Override Existing SP MgmtDetails
Node-1 SP MgmtDetails(Manual)
Node-2 SP MgmtDetails(Manual)
Node-3 SP MgmtDetails(Manual)
Node-4 SP MgmtDetails(Manual)
Node-5 SP MgmtDetails(Manual)
Node-6 SP MgmtDetails(Manual)
Node-7 SP MgmtDetails(Manual)
Node-8 SP MgmtDetails(Manual)
DNS Domain Names
DNS Server IP Addresses
NTP Primary
NTP Alternative(Optional)

Section: Support
AutoSupport Status
AutoSupport URL(Optional)
EMS Mail IDs
EMS Syslog Server
SysBackup Transport Protocol
SysBackup Destination Address