Tuesday, 28 March 2017

ONTAP 9.1 Guided Cluster Setup - ClusterSetupTemplate.CSV

I previously blogged about this here. Since I don’t always have access to a pristine 9.1 instance, and I’m terribly forgetful where I put stuff, I thought it would be helpful to detail the ClusterSetupTemplate.CSV further.

Contents

1) Tables
- Cluster Details
- Network Details 1 - General
- Network Details 2 - Cluster and Node Mgmt.
- Network Details 3 - SP Override
- Network Details 4 - SP
- Network Details 5 - DNS & NTP
- Support Details
2) Raw CSV in Text format

1) Tables

Image: Cluster Details

Image: Network Details 1 - General

Image: Network Details 2 - Cluster and Node Mgmt.

Image: Network Details 3 - SP Override

Image: Network Details 4 - SP

Image: Network Details 5 - DNS & NTP

Image: Support Details

2) Raw CSV in Text format

Copy and paste the below content into a text editor and save as ClusterSetupTemplate.CSV

Section,Field,Value,,Help Info
Cluster,Version,9.1,$$,
,Cluster Name,,$$,
,Configuration Type(for SingleNodeCluster),,$$,"To create a single-node cluster, type True.
If you type “True”, all the cluster LIFs will be deleted and all the ports in the cluster IP space will be moved to data.
To create a single-node cluster without deleting the cluster LIFs, type False.
This is applicable only when a single node is discovered."
,Configuration Type(for TwoNodeCluster),,$$,"Type switched/switchless based on your cluster configuration type.
Supported only for a two-node cluster. Ensure that the cluster configuration type is correct. If you have entered switched configuration and if your cluster configuration is switchless, the cluster will not be usable."
,Base License,,$$,"To create a cluster, the cluster base license key is optional.
If you want to provide the feature license key, then the base license key is mandatory. "
,Feature License(Optional),,$$,Provide semicolon separated values.
Network,Network Configuration Type,,$$,"To enter a range of IP addresses, type Range.
If you have the IP addresses in a sequence (for example, if you have an address range such as 10.10.10.10 through 10.10.10.15, then you can provide the IP address range as 10.10.10.10/10.10.10.15//.
If you have entered “Range” in the “Value” field, you must provide the IP address range in the “IP Address Range” fields.
To enter individual IP addresses, type Manual.
Provide the IP addresses for cluster management, node management, and service provider networks in the corresponding fields."
,IP Address Range1,,$$,"Provide the IP address range in the following format:
The IP addresses that you have provided will be applied sequentially."
,IP Address Range2,,$$,"Provide the IP address range in the following format:
The IP addresses that you have provided will be applied sequentially."
,IP Address Range3,,$$,"Provide the IP address range in the following format:
The IP addresses that you have provided will be applied sequentially."
,Cluster MgmtDetails(Manual),,$$,"Provide the cluster management details in the following format:
"
,Node-2 MgmtDetails(Manual),,$$,"Provide the node management details in the following format:
For an n-node cluster configuration, provide the node management details until the nth node."
,Node-3 MgmtDetails(Manual),,$$,"Provide the node management details in the following format:
"
,Node-4 MgmtDetails(Manual),,$$,"Provide the node management details in the following format:
"
,Node-5 MgmtDetails(Manual),,$$,"Provide the node management details in the following format:
"
,Node-6 MgmtDetails(Manual),,$$,"Provide the node management details in the following format:
"
,Node-7 MgmtDetails(Manual),,$$,"Provide the node management details in the following format:
"
,Node-8 MgmtDetails(Manual),,$$,"Provide the node management details in the following format:
"
,Override Existing SP MgmtDetails,,$$,"To override the existing Service Processor configuration, type True.
To retain the existing Service Processor configuration, type False.
If you have entered ""Manual"" in the ""Network Configuration Type"" list and you want to override existing Service Processor configurations, provide the Service Processor management details in the following rows.
If you have entered ""Range"" in the ""Network Configuration Type"" list and you want to override existing Service Processor configurations, the IP address range that you have provided will be applied sequentially for Service Processor management. "
,Node-1 SP MgmtDetails(Manual),,$$,"Provide the Service Processor management details in the following format:
For an n-node cluster configuration, provide the Service Processor management details up to the nth node. "
,Node-2 SP MgmtDetails(Manual),,$$,"Provide the Service Processor management details in the following format:
< Service Processor management IP address/Netmask/Gateway>"
,Node-3 SP MgmtDetails(Manual),,$$,"Provide the Service Processor management details in the following format:
< Service Processor management IP address/Netmask/Gateway>"
,Node-4 SP MgmtDetails(Manual),,$$,"Provide the Service Processor management details in the following format:
< Service Processor management IP address/Netmask/Gateway>"
,Node-5 SP MgmtDetails(Manual),,$$,"Provide the Service Processor management details in the following format:
< Service Processor management IP address/Netmask/Gateway>"
,Node-6 SP MgmtDetails(Manual),,$$,"Provide the Service Processor management details in the following format:
< Service Processor management IP address/Netmask/Gateway>"
,Node-7 SP MgmtDetails(Manual),,$$,"Provide the Service Processor management details in the following format:
< Service Processor management IP address/Netmask/Gateway>"
,Node-8 SP MgmtDetails(Manual),,$$,"Provide the Service Processor management details in the following format:
< Service Processor management IP address/Netmask/Gateway>"
,DNS Domain Names,,$$,"Provide semicolon separated DNS domain names.
You can provide up to six domain names."
,DNS Server IP Addresses,,$$,"Provide semicolon separated DNS server IP addresses. Only IPv4 addresses are supported.
You can provide up to three DNS server IP addresses."
,NTP Primary,,$$,Provide primary NTP server details.
,NTP Alternative(Optional),,$$,Provide alternative NTP server details.
Support,AutoSupport Status,,$$,"To enable AutoSupport on all the nodes, type True.
To disable AutoSupport on all the nodes, type False."
,AutoSupport URL(Optional),,$$,"If you have enabled AutoSupport, enter the AutoSupport URL in the following format:
username:password@proxyUrl:port"
,EMS SMTP MailHost,,$$,
,EMS Mail IDs,,$$,Provide semicolon separated email IDs.
,EMS SNMP TrapHost,,$$,
,EMS Syslog Server,,$$,
,SysBackup Transport Protocol,,$$,"Provide an appropriate transfer protocol for system backup (FTP or HTTP).
System backup is applicable only for single-node clusters."
,SysBackup Destination Address,,$$,"Provide a destination address for system backup.
System backup is applicable only for single-node clusters."

Sunday, 26 March 2017

Testing ONTAP Maximums with PowerShell: ‘Export Policy Rules’ Example

Quite possibly a pointless post - this has never been a good reason to stop me posting something…

I needed to test if there’s a limit to the number of export policy rules in an export policy.
If you check out the ONTAP Maximums, it says (this is for small (<=4) / medium (<=8) / large clusters) -

Maximum number of export policies = 4000 / 12000 / 12000
Maximum number of export rules = 70000 / 70000 / 140000

- it also says that ‘these aren’t enforced by code’, so we don’t really expect to hit any limit. The ONTAP Maximums don’t tell us “Maximum number of export rules in an export policy”, hence the test (the figures above were the same in 8.3.x as 9.1).

To find out if there is any enforced limit with number of export rules in an export policy, you can run a simple test in PowerShell with the Data ONTAP PowerShell toolkit.


Import-Module DataOntap
Connect-NcController CLUSTER_NAME_IP
New-NcExportPolicy -Name TEST1 -VserverContext SVM
for($i = 1; $i -lt 50000; $i++){ $i; New-NcExportRule -Policy TEST1 -VserverContext SVM -index $i -ClientMatch ("host" + $i + ".lab.priv") -ReadOnlySecurityFlavor any -ReadWriteSecurityFlavor sys}


I went for 50’000 and had no problem creating 50’000. It’s not a good idea to have anything like this number of export rules in an export policy (the use of NetGroups to manage sets of clients - where there’s lots of client - is highly recommended!) Really, I was just curious to see if 4’000 was possible (don’t ask why.)

Image: Export Policy with 50’000 Export Policy Rules!

Monday, 20 March 2017

NetApp CN1610 Cluster Switch Notes: How to Setup

I realized I’ve done loads of CN1610s but never blogged my notes (always referring to notes on my laptop), so here we go (cabling is not covered here, just switch setup.)

Image: NetApp CN1610 Cluster Switches with ISLs Cabled

1) Initial Cluster Switch Setup Script

Connect a laptop to the switch’s console (RJ45) port. Out-of-the-factory, the default CN1610 login is:

username = admin
password = {no password}

You will initially be in the user USER command mode: (CN1610) >
From here, copy and paste the below script, with the highlighted entries updated accordingly:


en
serviceport protocol none
y
network protocol none
y
serviceport ip SWITCH_IP NETWORK_MASK GATEWAY
hostname SWITCH_NAME
configure
username admin password PASSWORD
exit
write memory
y
reload


Repeat on both switches. To verify serviceport and network settings:


(CN1610) # show serviceport
(CN1610) # show network


Note: Type ‘en’ or ‘enable’ to get from the USER command mode - (CN1610) > - to the EXEC mode - (CN1610) #

2) Cluster Switch OS and RCF File

At the time of writing, the recommended versions of Fastpath OS and RCF File for CN1610 cluster switches with ONTAP 8.3 to 9.1 are:

OS FASTPATH Version = 1.2.0.7
RCF Version = 1.2

The Compatibility Matrix is at the link below (software and instructions are available off this link too):

To check FASTPATH and RCF (usually they’re correct out-of-the-factory), run the below commands - the RCF version is listed in the description for interface 3/64:


(CN1610) # show version
(CN1610) # show running-config


Upgrading FASTPATH and/or the RCF requires a TFTP server.

If you need to upgrade the RCF (NetApp recommends doing the RCF before OS), the commands are:


(CN1610) # show running-config config_backup.scr
(CN1610) # copy tftp://tftpserver/CN1610_CS_RCF_v1.2.txt nvram:script CN1610_CS_RCF_v1.2.scr
(CN1610) # script list
(CN1610) # script apply CN1610_CS_RCF_v1.2.scr
(CN1610) # show running-config
(CN1610) # write memory
(CN1610) # reload


If you need to upgrade FASTPATH, the commands are:


(CN1610) # show bootvar
(CN1610) # copy active backup
(CN1610) # show bootvar
(CN1610) # copy tftp://tftp_server_address/NetApp_CN1610_1.2.0.7.stk active
(CN1610) # reload

(CN1610) > enable
(CN1610) # show version


3) Configuring DNS, NTP, and SSH

Configure DNS, NTP and SSH using the commands below, with the highlighted entries updated accordingly:


(CN1610) # config
(CN1610) (Config)# ip domain name DNS_DOMAIN_NAME
(CN1610) (Config)# ip name server NAME_SERVER_1,NAME_SERVER_2,...
(CN1610) (Config)# sntp client mode unicast
(CN1610) (Config)# sntp server NTP_SERVER_1
(CN1610) (Config)# sntp server NTP_SERVER_2
(CN1610) (Config)# clock timezone +/- NUMBER*
(CN1610) (Config)# exit
(CN1610) # show ip name server
(CN1610) # show sntp

(CN1610) # ip ssh protocol 2
(CN1610) # config
(CN1610) (Config)# crypto key gen rsa
(CN1610) (Config)# crypto key gen dsa
(CN1610) (Config)# exit
(CN1610) # ip ssh server enable
(CN1610) # show ip ssh


*UTC 0 is 0, UTC -1 is -1, UTC +2 is 2, etcetera.
Note: SSH needs to be enabled before running Config Advisor. The RCF version 1.2 disables telnet (for security reasons) with the line "ip telnet server enable".

4) Passwords

To change the current logged in user’s password:


(CN1610) > password


To set an enable password:


(CN1610) # enable password PASSWORD


5) Configuring Switches for Email Alerting

Use the following commands starting from the EXEC privilege level (#). Replacing highlighted entries as required:


config
mail-server MAIL_SERVER
exit
logging email
logging email 3
logging email urgent 2
logging email message-type both to-addr MAIL_TO_ADDRESS
logging email from-addr MAIL_FROM_ADDRESS
logging email message-type both subject "Alert from SWITCH_NAME"
logging email test message-type both message-body "THIS IS A TEST"
exit
show logging email config
show running-config
write memory


To test email alerting from the EXEC privilege level (#):


configure
logging email test message-type both message-body "THIS IS A TEST - PLEASE IGNORE!"
exit
exit
logout


6) Saving Changes (IMPORTANT)

To save changes so that they are persistent to reboots:


(CN1610) # write memory



Saturday, 18 March 2017

Upgrade ONTAP 8.3.1 to 9.1 Walkthrough




Back in October I posted Cluster Software Update Essentials which outlined 3 different upgrade methods (really, methods 1 & 2 were the same, just GUI v CLI.) In this post I’m going to run through the method I’d use - “cluster image update” (for anything other than single node clusters.) Upgrading from 8.3.0 to 9.1 is a one step process.

Pre-Preparation

1) Verify your environment is supported for ONTAP 9.1
- and make remediation’s where required.
Note: This includes FC SAN hosts, boot from SAN hosts, OFFTAP products…
2) Verify platform support, and root volume sizes
3) Verify cluster health via AutoSupport
4) Verify cluster configuration with Config Advisor
5) Verify cluster health in OnCommand Unified Manager.
6) Thoroughly go over the Upgrade Advisor outputs and do the appropriate!
7) Download the ONTAP 9.1 image
8) Serve the 91P1_q_image.tgz file on a web server.

Note: 9.1P1 is the recommend release at the time of writing.

Cluster Image Package Get

Use the highlighted commands below to get the new image on the cluster, and perform validation:


cluster1::> cluster image show
Node        Current Version Installation Date
----------- --------------- -------------------
cluster1-01 8.3.1           9/17/2015 16:54:16
cluster1-02 8.3.1           9/17/2015 16:55:04

cluster1::> cluster image package show-repository
There are no packages in the repository.

cluster1::> cluster image package get -url http://192.168.0.5:8080/91P1_q_image.tgz

Software get http://192.168.0.5:8080/91P1_q_image.tgz started on node cluster1-01
Downloading package. This may take up to 10 minutes...

cluster1::> cluster image package show-repository
Package Version Package Build Time
--------------- ------------------
9.1P1           2/14/2017 13:14:46

cluster1::> cluster image validate -version 9.1P1

It can take several minutes to complete validation...

Pre-update Check                  Status
----------------------------      ------
Aggregate plex resync status      OK
Aggregate status                  OK
Autoboot Status                   OK
Broadcast Domain status           OK
CIFS status                       OK
CPU Utilization Status            OK
Cluster health status             OK
Cluster quorum status             OK
Data ONTAP Version Status         OK
Disk status                       OK
High Availability status          OK
Jobs Status                       OK
LIF failover                      OK
LIF load balancing                OK
LIFs not hosted                   OK
LIFs on home node status          OK
Manual checks                     Warning: Manual validation checks need to be performed. Refer to the Upgrade Advisor...
MetroCluster configuration status OK
NDMP status                       OK
NFS netgroup check                OK
Platform status                   OK
Previous Upgrade Status           OK
SAN LIF status                    OK
SAN status                        OK
Security Config SSLv3 check       OK
SnapMirror status                 OK
Snapshot copy count check         OK
Volume move status                OK
Volume status                     OK
Overall Status                    Warning


Update the first HA Pair

Note the following warning from Upgrade Advisor regards SnapMirror:
“To prevent SnapMirror transfers from failing, you must suspend (Quiesce) SnapMirror operations and upgrade destination nodes before upgrading source nodes.
(i) Suspend SnapMirror transfers for a destination volume
(ii) Upgrade the node that contains the destination volume
(iii) Upgrade the node that contains the source volume
(iv) Resume the SnapMirror transfers for the destination volume”

This is a good point at which to send AutoSupports (and verify they have sent)::>


autosupport invoke -node * -type all -message "maint=4h upgrading to ONTAP 9.1P1"
autosupport history show -node cluster1-01
autosupport history show -node cluster1-02


Note: If this is a 4-node or greater cluster, you might like to make sure Epsilon is not on the HA-Pair being upgraded (cluster image update will handle this.)


set adv
cluster show
cluster modify -node EPSILON_NODE -epsilon false
cluster modify -node NEW_EPSILON_NODE -epsilon true


Update the HA-Pair:


cluster1::> cluster image update -version 9.1P1 -nodes cluster1-02,cluster1-01

Starting validation for this update. Please wait...
It can take several minutes to complete validation...
Warning: Validation has reported warnings. Do you want to continue? {y|n}: y
Starting update...


Note: You don’t need to do 1 HA-pair at a time, “cluster image update” is designed to update the entire cluster.

Verifying the Update Process

You should be connected to Service-Processors to observe the update process.
Note: If you have aggregates serving CIFS, you may need to do a “storage failover giveback -ofnode NODE -override-vetoes true”


cluster image show-update-progress
storage failover show-takeover
storage failover show
storage failover show-giveback


Update Subsequent HA Pairs

Follow the above to update subsequent HA Pairs as required.

Finally

Verify the cluster version has increased (the cluster version only updates when all nodes are upgraded).
If you suspended SnapMirrors re-enable them.
Send AutoSupports::>


version
snapmirror resume -source-path {SRC_PATH} -destination-path {DEST_PATH}
autosupport invoke -node * -type all -message "maint=END finished upgrade to ONTAP 9.1P1"


And test your environment.

Appendix: (Some) Supported things with ONTAP 9.1

Just a personal reference to save some IMT results (correct at 2017.03.18):

OnCommand Unified Manager 7.0
OnCommand Unified Manager 7.1
OnCommand Performance Manager 7.0
OnCommand Performance Manager 7.1
SnapDrive 7.1.3 for Windows
SnapDrive 7.1.4 for Windows
SnapCenter Host Plug-in 1.1 for Microsoft Windows
SnapCenter Host Plug-in 1.1 for UNIX
SnapCenter Host Plug-in 2.0 for Microsoft Windows
SnapCenter Host Plug-in 2.0 for UNIX
SnapCenter Host Plug-in 2.0 for VMware vSphere
Virtual Storage Console 6.1
Virtual Storage Console 6.2
Virtual Storage Console 6.2.1
SnapDrive 5.2.2 for Unix*
SnapDrive 5.3 for Unix*
SnapDrive 5.3.1 for Unix*

SnapCenter Application Plugin has dependencies with:
- SnapCenter Host Plugin (if 1.1 use 1.1/if 2.0 use 2.0)
- SnapCenter Server
- Host Application

SnapCenter Server has dependencies with:
- SnapCenter Host Plugin (if 1.1 use 1.1/if 2.0 use 2.0)
- SnapCenter Application Plugin
- Virtual Storage Console (VMware)

SnapManager for Exchange has dependencies with:
- SnapDrive (for Windows version)
- Host OS (Windows Server version)
- Host Application (Exchange Server version)

CN1610 Cluster Switch FASTPATH 1.2.0.7 and RCF 1.2
Cisco NX3132V Cluster Switch NX-OS 7.0(3)I4(1) and RCF 1.1
Cisco NX5596 Cluster Switch NX-OS 7.1(1)N1(1) and RCF 1.3
Cisco NX5020 Cluster Switch NX-OS 5.2(1)N1(8b) and RCF 1.3
Cisco NX5010 Cluster Switch NX-OS 5.2(1)N1(8b) and RCF 1.3