Thursday, 13 December 2018

NetApp HCI Network Essentials

In order to successfully deploy NetApp HCI, it is essential that the network piece is correctly setup beforehand. The below links and diagrams are ones I’ve found helpful.

Switch Requirements for Deployment to be Successful:
(1.4) NetApp HCI Documentation Center
> Setting up and using NetApp HCI 1.4
>> Deploying NetApp HCI
>>> Requirements for NetApp HCI deployment
>>>> Network and switch requirements
(1.3.1) NetApp HCI Documentation Center
> Setting up and using NetApp HCI 1.3.1
>> Deploying NetApp HCI
>>> Requirements for NetApp HCI deployment
>>>> Network and switch requirements

Cisco document on how to set up the VPC:
Products & Services / Switches / Data Center Switches / Cisco Nexus 5000 Series Switches / White Papers

Some examples of an existing configuration:
NetApp HCI Documentation Center
> Setting up and using NetApp HCI 1.3.1
>> Deploying NetApp HCI
>>> Requirements for NetApp HCI deployment
>>>> Network and switch requirements
>>>>> Switch port configuration examples

NetApp HCI Network Setup Guide

The below diagrams come for the HCI Network Setup Guide - the important thing to note is that it is a requirement to have LACP (vPC) for the Storage Node Bond10G (technically, LACP is not a requirement, but it is highly recommended, and without it, performance will be poor - so don’t even consider it for HCI test/demo configurations.)

Note: The below might be slightly out of date for NDE 1.4.

Image 1/4: Storage Node Network Connections (Configure LACP for Bond10G)

Image 2/4: Storage Node VLAN Map

Image 3/4: Compute Node Network Connections (Do not configure LACP)

Image 4/4: Compute Node VLAN Map - Dedicated vMotion Network*

*vMotion can share the same network as management. vMotion cannot share the iSCSI subnet/VLAN.

Additional Links

NetApp HCI Documentation Resources

NetApp HCI Theory of Operations

Sunday, 25 November 2018

Creating Bootable USB Sticks to RTFI* NetApp HCI Nodes

*RTFI = Return to Factory Image

I needed to prep some USB sticks ready for an upcoming NetApp HCI install. There are various reasons why you might want to RFTI: reset a test/demo/lab environment, apply the latest versions, someone’s muddled the disks up...

Firstly, you will need a bootable USB drive maker. Go to:
And download rufus-2.6p.exe. I know 2.6 works for this purpose (I had issues with 3.3). The ‘p’ is for ‘portable’/don’t-need-to-install which I prefer, but you don’t have to go for the ‘p’ version.

Secondly, you will need to get the NetApp HCI compute and storage node images from (Product = ‘NetApp HCI’):

The ISOs are quite big. Storage node ISOs are ~4GB and Computes node ISOs ~ 15GB (mainly because it contains a few different versions of VMware vSphere).

Here I’m using NetApp HCI 1.3.1 ISOs:
- solidfire-rtfi-neon-patch3- (Storage Node)
- solidfire-NetApp-HCI-Bootstrap-OS- (Compute Node)

Image: NetApp HCI Storage and Compute Node ISOs

Step 1) Double-Click rufus-2.6p.exe to run

Image: rufus-2.6p.exe

Step 2) Storage Node

Step 2.1) Enter settings into Rufus as thus:

First check ‘Create a bootable disk using...’ and click the CD icon to the right, and point to the solidfire-rtfi-neon-patch3- (or whatever version you’re using).

Device = {Your USB Key}
Partition scheme = MBR partition scheme for BIOS or UEFI-CSM
File system = FAT32 (Default)
Cluster size = 8192 bytes
New volume label = solidfire-rtfi
Uncheck ‘Check device for bad blocks’
Check ‘Quick format’
Already checked ‘Create a bootable disk using...’
Check ‘Create extended label and icon files’

Image: Rufus Settings for Storage Node

And click Start

Step 2.2) Select ‘Write in ISO Image mode’

Image: ISOHybrid image detected - Write in ISO Image Mode

Step 2.3) Click OK to the warning


Step 2.4) Wait for Rufus to finish!

Image: Rufus done!

This will probably take about 5 minutes.

Step 3) Compute Node

Setting up Rufus for the Compute Node RTFI USB drive is pretty much the same as for the Storage Node, except you must choose ‘Write in DD Image mode’ (IMPORTANT!)

You will need to point to the solidfire-NetApp-HCI-Bootstrap-OS- (or whatever version you’re using).
I let the ‘New volume label’ autopopulate with ember-hci...

Image: Rufus Settings for Compute Node

Image: ISOHybrid image detected - Write in DD Image Mode

It took a good 40 minutes (on my system) to create the Compute Node RTFI USB drive.

Bonus Material 1 of 2: Correct Syslinux Version

One problem you’ll have with Rufus 2.6, is that it cannot download the correct versions of Syslinux, and you’ll get “The requested item could not be located”. My trick was to use Rufus 3.3 to download the correct versions of Syslinux.

Image: Syslinux 5.10/2013-06-04 is required

Image: Rufus 2.6.818 - “The requested item could not be located.”

Bonus Material 2 of 2: Why Not Use Rufus 3.3?

Rufus 3.3 works fine for the Storage Node, but not the Compute Node. Trying to create the USB key for the Compute Node, you’ll get stuck because you can’t select anything other than NTFS for File System, and when you click START, get the error message: “Unsupported filesystem: The file system currently selected can not be used with this type of ISO. Please select a different file system or use a different ISO.”

Image: Rufus 3.3.1400 - “Unsupported Filesystem”

Tuesday, 20 November 2018

If you’re headswapping to AFF A700 / A700s you might need more disk...

Some of the original NetApp AFF (All Flash FAS) platforms are reaching the age where one might consider headswapping them. One of the first things we check before any headswap (this was the same for 7-Mode headswaps) is that the root volume is of sufficient size to support the new platform, and if the root volume is too small, we make it bigger before commencing the headswap.

The AFF8020 / AFF8040 / AFF8060 / AFF8080EX have min. root volume size of 350 GiB.
The AFF A200 / A300 have min. root volume size of 350 GiB.
The AFF A700s / A700 have min. root volume size of 934 Gib / 962 Gib.

So, if you want to headswap to an AFF A700s / A700, and you had ADP RD2 partitioned root aggregates, and the root aggregate size is less than the minimum root aggregate size - which is 1 TiB - for the AFF A700s / A700, then you might need to add more disk so can you can create a root aggregate big enough to hold the root volume.

Image: HWU for NetApp AFFs with ONTAP 9.3P7-P9

Saturday, 17 November 2018

NetApp HCI: Some Notes, References and Links

I‘ve not posted anything about NetApp HCI since July 2017 (my first and only dedicated post), so about time to update!
Note: The below is correct at time of writing (mid-November 2018)

NetApp HCI Installation Workbook
This is something a customer will be sent prior to a NetApp/Partner Professional Services installation engagement. Essentially it is to make sure the environment and required information (network switches, DNS, NTP, IP address, VLANs, etcetera...) is ready prior to the PS Engineer arriving on site.

NetApp HCI Core Software
The HCI software is pre-loaded on the nodes, but if they shipped on a previous version and you want the latest, or perhaps you’re testing and want to blat it to start anew:
From the above you can download:
Software download for NetApp HCI compute node:
Software download for NetApp HCI storage nodes, Element OS:
Software download for NetApp HCI management node:

(Other) NetApp HCI Software

NetApp HCI Documentation
Key documents:

NetApp Interoperability Matrix Tool
Solution Search > Converged Infrastructure > NetApp HCI
Briefly - supported includes:
- VMware ESXi 6.0 U3 / 6.5 U1 / 6.5 U2 / 6.7
- Element OS 10.0 / 10.1 / 10.2 / 10.3 / 10.4
*See VMware HCL for all Guest OSes that are supported.

Blogs on NetApp HCI
NetApp HCI: It’s So Easy, It’s Almost Anti-Climactic
HCI - Hero from Day Zero
NetApp HCI: The first installation. The force awakens (and it glows)
NetApp HCI: Your Biggest Questions Answered

NetApp HCI YouTube Videos

NetApp TRs and NVAs
General Technical Reports:
Solution Specific Technical Reports and NetApp Verified Architectures

If you have a need to RTFI the nodes (i.e. a new version of the NDE has come out and you want to use the latest), then go to and search for RTFI. To create bootable USB keys you can use the Rufus tool from (can RTFI from the IPMIs also).

Note: If I get around to it I'll do a post on preparing bootable USB keys. The process is a little different for the compute and storage nodes.

Image: After racking, cabling, and powering, just this left to do (from the ISI)

Other Blogs
What is the NetApp Solidfire / HCI mNode?
NetApp SolidFire mNode and ActiveIQ Monitoring

Additionally, if you’re NetApp/Partner, a few extra resources:
1) check out the ‘NetApp HCI Product Page
This includes additional brainshark videos:
Also a Technical Presentation and ‘POC/Test Plan
2) The HCI PS Experts Team Sharepoint page:
Here you will find the HCI_Installation_Workbook_v2.1.xlsm amongst other things.
3) I can't remember where I got this from, but someone did a nice 'HCI SETUP - DEMO.docx'
4) Another interesting document - which came from a HCI Bootcamp - 'HCI_Installation_Process_v1.0_Generic.docx'

Sunday, 11 November 2018

NetApp Storage Encryption (NSE) Researches (Specifically with Regards to Upgrading from Clustered ONTAP 8.2.1)

Some NSE (NetApp Storage Encryption) links that may be useful if you’re looking to upgrade a system with NSE disks from 8.2.1 to 9.3. A typical upgrade path would be 8.2.1 -> 8.3.2 -> 9.1 -> 9.3 (the minimum recommended P-releases at the time of writing are 8.3.2P12, 9.1P15, 9.3P8).

How to upgrade Data ONTAP 8.3 (or prior) to 8.3.1 or later that is using NetApp Storage Encryption

NSE: How to upgrade a NSE controller with external key management (KMIP) server to ONTAP 9.3 or later

NetApp IMT
To check your solution is supported with the various ONTAP versions, search for:
Storage Solution = Key Managers

NetApp Hardware Universe
Check your platform supports the version of ONTAP you want to go to.

TR-4074: NetApp Storage Encryption: Preinstallation Requirements and Procedures for SafeNet KeySecure

Setting up Storage Encryption (in 8.2.1)

Managing Storage Encryption (in 8.2.1)

(PDF) Clustered Data ONTAP 8.2 Physical Storage Management Guide (Updated for 8.2.1)
Similar to the above two links but the PDF version. Check from page 74 “Managing Storage Encryption”.

How to configure NSE in clustered Data ONTAP 8.3.1 and later

(GPS) NetApp Storage Encryption NSE

Services Partners: NSE - How to boot NSE when Key Servers are down or unreachable

Other non-NSE Specific Stuff (Upgrade related stuff)

Clustered Data ONTAP 8.3.2 Upgrade and Revert/Downgrade Guide

{EITHER} How to Check Data ONTAP 8.3.2 Upgrade Requirements Using A PowerShell Script

{AND/OR} "Steps for preparing for a major upgrade" ~ pages 32-68

Install Validation Failed. ERROR: LIF sufficiency check failed

ONTAP 9.1 Upgrade and Revert/Downgrade Guide

Upgrading Clustered Data ONTAP 8.3x To ONTAP 9.1 Using Automated Nondisruptive Upgrade Method