MetroCluster Installation Workflow

Just some notes from ‘Planning to Implement MetroCluster on Clustered Data ONTAP’. The web based course was a bit old - ONTAP 8.3.1 - but still useful as a refresher/framework.

1) Prepare for the MetroCluster Installation

Q1: What components do we have / do we have all the required components?

Obtain SO/BoM

MetroCluster Configurations:

i) Stretch Configurations
- 2-Node direct-attached (with SAS cables)
- 2-Node bridge-attached (with ATTO bridges)
ii) Fabric-Attached Configurations
- 2-Node (2 x single-node clusters)
- 4-Node (2 x 1 HA-pair clusters)
- 8-Node (2 x 2 HA-pair clusters)

High-Level Components List of 4-Node Fabric-Attached MetroCluster:

     |- Cluster Peering Network
     |- ISLs*
#####|Cluster A Site A
     |- 2 x NetApp FAS/AFF Controllers (HA-Pair)
     |- Cluster ISLs
     |- 2 x Fabric Switches
     |- Storage Shelves
     |- Fiber Channel-to-SAS bridges**
#####|Cluster B Site B
     |- 2 x NetApp FAS/AFF Controllers (HA-Pair)
     |- Cluster ISLs
     |- 2 x Fabric Switches
     |- Storage Shelves
     |- Fiber Channel-to-SAS bridges**

* 2 to 4 per fabric
** 2 per stack

Image: Differences between Clustered Data ONTAP MetroCluster Configurations (in 8.3.1)
* Requires Tie Breaker and PVR

Q2: What software versions are we using?

     |Fabric Switch Firmware
     |ATTO Bridge Firmware

Download required softwares
> Fibre Channel Switch
> Protocol Bridge (for ATTOs)
Download FC Switch Golden Configuration Image files
Download ‘ATTO QuickNAV utility’ to setup ATTOs

Q3: Have we obtained the Hardware Setup Checklist and Software Setup Checklist?

Select your ONTAP version and look for the:
‘MetroCluster Installation and Configuration Guide’

Q4: Device names?

Image: MetroCluster Naming Conventions
2) Configure the MetroCluster Hardware Components

See the ‘MetroCluster Installation and Configuration Guide’

2.1) Install Components and cable the storage controllers
- Rack the components
- Cable the FC-VI and HBA adapters
- Cable the ISLs between MetroCluster sites
- Cable the Cluster Interconnect
- Cable the Cluster Peering connections
- Cable the HA Interconnect
- Cable the Management connections
- Cable the Data ports
- Review the port connections
2.2) Cable and configure storage
- Set shelf IDs
- Configure ATTOs
ATTO Notes:
- Each ATTO bridge must have a unique name
- Configure data rate
- Must set the bridge connection mode to PTP
- Update Firmware
- FibreBridge Installation and Operation Manual available on the ATTO website
ATTO> sastargets
2.3) Configure the FC switches
- FTP Golden Configuration Image files to correct switch
2.4) Repeat for partner cluster

Image: Switch configuration by the NetApp Golden Configuration Image File (Brocade 6505 and 6510)
3) Configure the MetroCluster Software Components

Note: Systems configured in the factory will have much of the below completed.

3.1) Gather the required information
- Switch information
- Cluster creation information
- Node information
- Logical interfaces (LIFs) and ports for cluster peering
- Time server information
- AutoSupport support tool information
- Service processor information
3.2) Verify the disk assignments
Check in Maintenance Mode.
*> storage show shelf
*> storage show disk
*> disk assign -shelf FC_switch_A_1:6.shelf10 -p 0
*> disk assign -shelf FC_switch_B_1:6.shelf20 -p 1
*> disk show -n
*> disk show
3.3) Verify the high-availability (HA) state
*> ha-config show # Should be mcc for all components in 4-node MC, or mcc-2n for 2-node MC
*> ha-config modify controller mcc
*> ha-config modify chassis mcc
3.4) Configure the clusters
- Power up each node
- Enable AutoSupport
- Configure Node Management
::> storage failover show
::> storage failover modify -node localhost -mode ha # If not configured in HA mode
::> net port show -role cluster
::> set adv
::*> network options switchless-cluster modify -enable true # If required
::*> set adm
Note: Recommendation is to use the ‘System Setup tool’ to create the cluster, but do not create aggregates! Or you can use CLI.
3.5) Implement the MetroCluster configuration
- Configure Intercluster LIFs
- Configure network route
CLU1::> cluster peer create -peer-addrs IP1,IP2
CLU2::> cluster peer create -peer-addrs IP1,IP2
::> cluster peer show -instance
::> cluster peer health show
CLU1::> storage aggregate mirror aggr0_controller_A_1
{and repeat for every root aggregate}
CLU1::> storage aggregate create aggr1_controller_A_1 -diskcount X -node controller_A_1 -mirror true
{and repeat for every node - creating a mirrored data aggregate}
::> storage aggregate show-status -aggregate AGGR_NAME
i) Make sure there are 2 non-root mirrored data aggregates present on each cluster (MCC-4 node)
ii) Make sure ha-config state of controller and chassis is mcc (MCC 4 node)> ha-config show
::> metrocluster configure -node-name {NODE_NAME}
Note: This enables metrocluster on all the nodes in the DR group that contains NODE_NAME.
::> metrocluster show
To implement MetroCluster using a Single Data Aggregate
::> set adv
::*> metrocluster configure -allow-with-one-aggregate-true {NODE_NAME}
::*> set adm

4) Prepare to Monitor and Manage the ONTAP MetroCluster Solution

::> metrocluster check run
::> metrocluster check show
::> metrocluster check config-replication show
::> metrocluster check config-replication show -instance
::> metrocluster check config-replication show-aggregate-eligibility
::> metrocluster check node show
::> metrocluster check aggregate show
::> metrocluster check config-replication capture-in-progress show
::> metrocluster check lif show

Run Config Advisor

Send ASUP before testing (on both clusters):
::> system node autosupport invoke -node * -type all -message "Maint=START"

Verify Local HA Operation:
::> storage failover show
::> storage failover takeover controller_A_2
::> storage failover show
::> storage failover giveback controller_A_2
... and repeat ...

Verify Switchover, Healing, and Switchback:
- Run the switchover command
- Confirm that disaster recovery partners have come on line
- Heal the configuration
- Heal the data aggregates
- Heal the root aggregates
- Perform a switchback

Configure the MetroCluster FC Switches for Health Monitoring:
::> storage switch add -switch-ipaddress X.X.X.X
::> storage switch show

Configure the FC-to-SAS Switches for Health Monitoring:
::> storage bridge add -address Y.Y.Y.Y
::> storage bridge show

Protecting Configuration Backup Files Against Disaster (essential for single-node clusters):
::*> system configuration backup settings modify -destination ftp://Z.Z.Z.Z -username admin
::*> system configuration backup settings show

Note 1: Job schedules on both clusters in a MetroCluster configuration must be identical.
Note 2: Both sites of the MetroCluster should be licensed for the same node-locked features.


  1. Vidad, you writes manual Installing the NetApp Software Stack on Windows Server 2008 R2 SP1.doc. how about NetApp Software Stack on Windows Server 2012 R2??


Post a Comment