Sunday, 8 July 2018

Difference Cisco Nexus 3132Q-V Rev Air vs Std Air

Everyone probably already knows this, but I didn’t until a few weeks ago.

The Blue tabs on fans on the NX3132Q-V Std Air, means the fan side should be in the cold (blue) aisle (symbols on the tab show air being sucked in.)

The Red tabs on fans on the NX3132Q-V Rev Air, means the fans blow hot air out the back and should be in the hot (red) aisle (symbols on the tab show air being blown out.)

Most commonly, the NX3132Q-V Std Air is used for the NetApp ONTAP cluster switches, since it makes the cabling easier (the ports at the back of the controllers are the same side of the cabinet as the switch ports.)

Image: Cisco Nexus 3132Q-V Std Air with Blue Tabs on Fans

Image: Cisco Nexus 3132Q-V Rev Air with Red Tabs on Fans

Sunday, 1 July 2018

MetroCluster IP Installation: Cabling the IP Switches: AFF A800 Systems

The following Visio diagrams are constructed from the MetroCluster IP Installation and Configuration Guide and the section:

Configuring the MetroCluster hardware components
> Installing and cabling MetroCluster components
> Cabling the IP switches

I’ve used the Cisco Nexus 3132Q-V switch in the diagrams.
I’m only showing cabling for the two switches in site A (SW_A_1 and SW_A_2).
These diagrams show cabling to AFF A800 systems.

Note from the manual regards MetroCluster ISLs:

Cable the switch ISL connections.
- One, two or three 40-Gbps ISLs are supported or up to six 10-Gbps MetroCluster ISLs.
- If using the Cisco 3232C switch in breakout mode, ports 21 - 24 are used as MetroCluster ISLs. In this case these 40-Gbps ports are split into four 10-Gbps ports. You must be using the correct RCF files to support the breakout configuration.
- The switch cannot be configured with both 40-Gbps and 10-Gbps ports.

Image: Legend

Image: Switch Site A Number 1 (SW_A_1) cabling to AFF A800

Image: Switch Site A Number 2 (SW_A_2) cabling to AFF A800

MetroCluster IP Installation: Cabling the IP Switches: AFF A700 and FAS9000 Systems

The following Visio diagrams are constructed from the MetroCluster IP Installation and Configuration Guide and the section:

Configuring the MetroCluster hardware components
> Installing and cabling MetroCluster components
> Cabling the IP switches

I’ve used the Cisco Nexus 3132Q-V switch in the diagrams.
I’m only showing cabling for the two switches in site A (SW_A_1 and SW_A_2).
These diagrams show cabling to AFF A700 and FAS9000 systems.

Note from the manual regards MetroCluster ISLs:

Cable the switch ISL connections.
- One, two or three 40-Gbps ISLs are supported or up to six 10-Gbps MetroCluster ISLs.
- If using the Cisco 3232C switch in breakout mode, ports 21 - 24 are used as MetroCluster ISLs. In this case these 40-Gbps ports are split into four 10-Gbps ports. You must be using the correct RCF files to support the breakout configuration.
- The switch cannot be configured with both 40-Gbps and 10-Gbps ports.

Image: Legend

Image: Switch Site A Number 1 (SW_A_1) cabling to AFF A700/FAS9000

Image: Switch Site A Number 2 (SW_A_2) cabling to AFF A700/FAS9000

Tuesday, 26 June 2018

How to Setup Cisco Nexus 3132Q-V NetApp ONTAP Cluster Switches

Back in April I posted Setting up Cisco Nexus 3132Q-V Switches for NetApp ONTAP. At the time I didn’t have access to a NX3132Q-V so had to source my information from manuals. In this post I post the console output from a real-world NX3132Q-V basic setup (came with the correct NXOS 7.0(3)I4(1) from the factory). The output is necessarily abridged and some details are changed to anonymize the output.

Note: These are NetApp provided NX3132Q-V switches - provided for the purpose of NetApp ONTAP cluster network.

First thing to do is connect up the console cable to your laptop, open a suitable client to view the console output (like PuTTY) and power up the switch.

First prompt you’ll see:


Abort Auto Provisioning and continue with normal setup ?(yes/no)[n]: yes


System Admin Account Setup:


         ---- System Admin Account Setup ----

Do you want to enforce secure password standard (yes/no) [y]: yes

    Enter the password for "admin": ********
  Confirm the password for "admin": ********


Note: Password should contain characters from at least three of the following classes: lower case letters, upper case letters, digits and special characters.

Basic System Configuration Dialog:


         ---- Basic System Configuration Dialog VDC: 1 ----

This setup utility will guide you through the basic configuration of the system. Setup configures only enough connectivity for management of the system.

...

Press Enter at anytime to skip a dialog. Use ctrl-c at anytime to skip the remaining dialogs.

Would you like to enter the basic configuration dialog (yes/no): yes

  Create another login account (yes/no) [n]: no
  Configure read-only SNMP community string (yes/no) [n]: no
  Configure read-write SNMP community string (yes/no) [n]: no
  Enter the switch name: SWITCHNAME
  Continue with Out-of-band (mgmt0) management configuration? (yes/no) [y]: yes
    Mgmt0 IPv4 address: A.A.A.A
    Mgmt0 IPv4 netmask: N.N.N.N
  Configure the default gateway? (yes/no) [y]: yes
    IPv4 address of the default gateway: G.G.G.G
  Configure advanced IP options? (yes/no) [n]: no
  Enable the telnet service? (yes/no) [n]: no
  Enable the ssh service? (yes/no) [y]: yes
    Type of ssh key you would like to generate (dsa/rsa) [rsa]: rsa
    Number of rsa key bits 1024-2048 [1024]: 1024
  Configure the ntp server? (yes/no) [n]: no
  Configure default interface layer (L3/L2) [L2]: L2
  Configure default switchport interface state (shut/noshut) [noshut]: noshut
  Configure CoPP system profile (strict/moderate/lenient/dense) [strict]: strict

The following configuration will be applied:

  password strength-check
  switchname SWITCHNAME
vrf context management
ip route 0.0.0.0/0 G.G.G.G
exit
  no feature telnet
  ssh key rsa 1024 force
  feature ssh
  system default switchport
  no system default switchport shutdown
  copp profile strict
interface mgmt0
ip address A.A.A.A N.N.N.N
no shutdown

Would you like to edit the configuration? (yes/no) [n]: no
Use this configuration and save it? (yes/no) [y]: yes


Then login to the switch:


User Access Verification
SWITCHNAME login: admin
Password: ********


Check the version is correct:


SWITCHNAME# show version

...

Software
  BIOS: version 04.24
  NXOS: version 7.0(3)I4(1)

...


The correct NX3132 RCF downloaded from the NetApp support site, needs to be applied. Here we download and apply the default RCF:


SWITCHNAME# copy tftp: bootflash: vrf management
Enter source filename: NX3132_RCF_v1.1_24p10g_26p40g.txt
Enter hostname for the tftp server: T.T.T.T
Trying to connect to tftp server......
Connection to Server Established.
TFTP get operation was successful
Copy complete, now saving to disk (please wait)...

SWITCHNAME# dir bootflash:
...
       8088    Jun 07 09:15:28 2018  NX3132_RCF_v1.1_24p10g_26p40g.txt
...

SWITCHNAME# copy bootflash:NX3132_RCF_v1.1_24p10g_26p40g.txt running-config

...

Copy complete.


Verify the config, copy to the startup config, then reboot the switch:


SWITCHNAME# show running-config
SWITCHNAME# copy running-config startup-config
SWITCHNAME# reload
This command will reboot the system. (y/n)?  [n] yes


If the RCF has been applied correctly, on reboot you will see (for the default RCF NX3132_RCF_v1.1_24p10g_26p40g.txt):


*****************************************************************************************
*  N3K NetApp Reference Configuration File (RCF) version 1.1-24p10g-26p40g (2016-08-23) *
*  40G ports 1/1 - 1/6 are configured as 4x10G ports 1/1-6/1-4.                         *
*                                                                                       *
*  Port 1 multiplexed H/W configuration options:                                        *
*    hardware profile front portmode qsfp      (40G H/W port 1/1 is active - default)   *
*    hardware profile front portmode sfp-plus  (10G H/W ports 1/1/1 - 1/1/4 are active) *
*    hardware profile front portmode qsfp      (To reset to QSFP)                       *
*****************************************************************************************


That’s it, basic configuration of a Cisco Nexus 3132Q-V NetApp ONTAP cluster switch is complete!

Image: Random bonus bit of information. The blue fins on the fans indicate that that side should be in the cold (blue) aisle.

Sunday, 27 May 2018

Tech Roundup - 27th May 2018

Stuff collected in the last month and a half since the last Tech Roundup. With headings:
AWS, Broadcom, Docker, Microsoft, NetApp, PowerShell, Storage Industry News, VMware, Miscellaneous

AWS

Amazon Web Services - Builders' Day London 2018 & Keynote presentation

Broadcom

Broadcom, NetApp & SUSE Announce Production Availability of the Industry’s First End-to-End NVMe over Fibre Channel Solution Enabling Groundbreaking Application Performance

Docker

Unveiling Docker Enterprise Edition 2.0:

Microsoft

ReFS is fully supported on ANY storage hardware that is listed on Microsoft HCL!
Note: The main obstacle for ReFS on SAN, is ReFS not yet having ODX and UNMAP implemented for SAN volumes, meaning these configurations are not HLK tested or certified. So, if you do need to use ODX, TRIM/UNMAP or Thin Provisioning for the underlying SAN LUNs, you should continue to use NTFS. Microsoft recommends turning these features off with ReFS to avoid potential issues. But some SANs enable thin provisioning by default, so it is important to keep this current restriction in mind.

Windows Admin Center aka Project "Honolulu" is now generally available!

Image: Windows Admin Center

For the first time ever, Microsoft will distribute its own version of Linux

NetApp

NetApp HCI Total Cost of Ownership (TCO) Study

Converged Versus Hyper Converged: What’s the Difference and What’s Right for Me?

How to Install and Scale Your NetApp SolidFire Cluster

How to Use Software Defined Storage in a ROBO Environment

10,000 Reasons Why You Need a Salesforce Backup

How Digital Transformation Paves the Way for Cloud-First Architectures

Great coverage on NetApp at Dell EMC World

SnapCenter YouTube Playlist

Deep Learning in Action: How Vincent Learned to Paint Like Van Gogh

Why SAP HANA Runs Better on NetApp: SAP System Refresh

NetApp: Spring Launch - May 2018

NetApp Cloud Volumes for Google Cloud Platform Strengthens Cloud Data Services Portfolio

Listed Alphabetically:

- 100GbE Cluster Interconnect and MetroCluster Switch
- Active IQ Services: New cloud-based predictive analytics services that deliver additional performance and data protection insights to customers.
- AFF A800: The industry’s first end-to-end NVMe-based enterprise all-flash system.
- AltaVault 4.4.1
- AFF A200
- Brocade 64-Port 32Gbps Blade
- Brocade G630 32GB Switch
- Digital Support 2.0: Our SupportEdge services combine live, cloud, and digital resources to deliver comprehensive support through Elio with AI powered predictive analytics.
- E-Series EF280
- FAS2720/FAS2750
- NetApp Manageability SDK 9.4
- OnCommand Unified Manager 9.4
- ONTAP 9.4: Power the newest AI and enterprise workloads with NVMe-accelerated performance, advanced cloud integration, and enhanced security features.
- SolidFire 18.2 Stack
- SolidFire VCP 4.0
- SSDs: The industry’s first 30TB SSDs: Shrink data center footprint and lower data center costs by storing over 2PB in a 2U shelf.
- StorageGRID 11.1: The number one data management solution for distributed organizations, now automates tamper proof retention of critical financial and personal data.

NetApp: The Pub

CloudMirror replication between StorageGRID WebScale and AWS

NetApp, Cisco and Red Hat announce OpenStack on FlexPod SolidFire!

Ansible support receives massively expanded ONTAP modules

Anonymous/Public Bucket Access in StorageGRID

Hear Ye! Hear Ye! Trident 18.04 is available now!

NetApp: Tech ONTAP Podcast

Episode 140: Quarterly Security Update - ONTAP 9.4 and GDPR

Episode 139: ONTAP 9.4 - NVMe and New Platforms

Episode 138: ONTAP 9.4 Overview

ONTAP 9.4 - Feature Overview

Episode 137: Name Services in ONTAP

Episode 136: Modernizing Dev and QA the NetApp Way

NetApp: TRs



PowerShell

Script Duplicate File Finder and Remover

Storage Industry News

Worldwide Converged Systems Revenue Increased 9.1% During the Fourth Quarter of 2017 with Vendor Revenue Reaching $3.6 Billion, According to IDC

VMware

VMware vSphere 6.7 is GA!

Windows 7 and 2008 virtual machines lose network connectivity on VMware Tools 10.2.0 (54483)

Miscellaneous

Logging with timestamp:

CLU::> set -prompt-timestamp inline

Image: Very Good Putty Session Logging Settings

Sunday, 6 May 2018

System Node Migrate-Root - Experience and Tips

I needed to test out ‘system node migrate-root’ in preparation for potentially using it prior to performing a headswap. No physical hardware here just an ONTAP 9.3 HA Simulator lab. These are some observations with a couple of tips.

Documentation

System Node Migrate-Root is documented here:

The syntax is very simple, for example:


cluster1::*>  system node migrate-root -node cluster1-01 -disklist VMw-1.17,VMw-1.18,VMw-1.19 -raid-type raid_dp

Warning: This operation will create a new root aggregate and replace the existing root on the node "cluster1-01". The existing root aggregate will be discarded.
Do you want to continue? {y|n}: y

Info: Started migrate-root job. Run "job show -id 86 -instance" command to
      check the progress of the job.


The process (see below) starts straight away.

The official documentation mentions:

The command starts a job that backs up the node configuration, creates a new aggregate, set it as new root aggregate, restores the node configuration and restores the names of original aggregate and volume. The job might take as long as a few hours depending on time it takes for zeroing the disks, rebooting the node and restoring the node configuration.

The Process and Timings

The SIM has tiny disks of 10.66GB and 28.44GB usable size. The 10.66GB disks were originally used for 3 disk root aggregates, and the migrate-root moved the root aggregate to the slightly larger virtual disks. On a physical system with much bigger than 28.44GB disks, I would expect the timings to be considerably longer than the below. The below timings are taken from acquiring the ‘Execution Progress’ string - from the job show output - every second.

0-27 seconds: Starting node configuration backup. This might take several minutes.
28-146 seconds: Starting aggregate relocation on the node "cluster1-02"
147-212 seconds: Rebooting the node to create a new root aggregate.
213-564 seconds: Waiting for the node to create a new root and come online. This might take a few minutes.
565-682 seconds: Making the old root aggregate online.
683-686 seconds: Copying contents from old root volume to new root volume.
687-864 seconds: Starting removal of old aggregate and volume and renaming the new root.
865-1653 seconds: Starting node configuration restore.
1654-1772 seconds: Enabling HA and relocating the aggregates. This might take a few minutes.
1773 seconds: Complete: Root aggregate migration successfully completed [0]

Nearly 30 minutes for migrate-root on one node of a tiny ONTAP 9.3RC1 HA SIM! And you still need to do a takeover/giveback of the node whose root aggregate was moved (see below).

Tips

1) The process disables HA and Storage Failover, and Aggregate Relocation is used to move the data aggregates to the node that’s staying up. The process does not move data LIFs, these will failover automatically, but I noticed a bit of a delay (my test CIFS share was down for 45 seconds), so I’d recommend moving data LIFs onto the node that’s going to stay up first.

2) I noticed - consistently - that if you run ‘system health alert show’ after the migrate-root completes, you get some weird output. Perform a takeover/giveback of the affected node after the migrate-root completes to correct this.


cluster1::*> system health alert show
This table is currently empty.

Warning: Unable to list entries for schm on node "cluster1-02": RPC: Remote
         system error [from mgwd on node "cluster1-02" (VSID: -1) to schm at
         127.0.0.1].
         Unable to list entries for shm on node "cluster1-02": RPC: Remote
         system error [from mgwd on node "cluster1-02" (VSID: -1) to shm at
         127.0.0.1].
         Unable to list entries for cphm on node "cluster1-02": RPC: Remote
         system error [from mgwd on node "cluster1-02" (VSID: -1) to cphm at
         127.0.0.1].
         Unable to list entries for cshm on node "cluster1-02": RPC: Remote
         system error [from mgwd on node "cluster1-02" (VSID: -1) to cshm at
         127.0.0.1].

cluster1::*> Replaying takeover WAFL log
May 06 14:01:04 [cluster1-01:monitor.globalStatus.critical:EMERGENCY]: This node has taken over cluster1-02.

cluster1::*> system health alert show
This table is currently empty.

cluster1::*>


Image: Remote system error [from mgwd on node...

PowerShell to Record Job Execution Progress Per Second

Sometimes when you’re doing stuff with NetApp ONTAP, you’ll get given a job-id and you’ll be curious to know all the phases the job goes through. So, I wrote this little program to find out just that. In the image - down the bottom - is my use case (which will be the subject of the next blog post.)


############################
## RecordJobExecution.ps1 ##
############################

Import-Module DataONTAP
$C = Read-Host "Enter Cluster FQDN/IP"
[Void](Connect-NcController $C)
$J = Read-Host "Enter Job ID"
while($TRUE){
  $D = Get-Date -uformat %Y%m%d
  $T = Get-Date -uFormat %T
  $P = (Get-NcJob -id $J).JobProgress
  "$D $T $P" >> execution_log.txt
  sleep 1
}


Image: Use Case: Recording per second the job execution progress of system node migrate-root