MetroCluster Installation Workflow

Just some notes from ‘Planning to Implement MetroCluster on Clustered Data ONTAP’. The web based course was a bit old - ONTAP 8.3.1 - but still useful as a refresher/framework.

1) Prepare for the MetroCluster Installation

Q1: What components do we have / do we have all the required components?

Obtain SO/BoM

MetroCluster Configurations:

i) Stretch Configurations

- 2-Node direct-attached (with SAS cables)

- 2-Node bridge-attached (with ATTO bridges)

ii) Fabric-Attached Configurations

- 2-Node (2 x single-node clusters)

- 4-Node (2 x 1 HA-pair clusters)

- 8-Node (2 x 2 HA-pair clusters)

High-Level Components List of 4-Node Fabric-Attached MetroCluster:

CHECK|COMPONENT

-----+----------

#####|Customer-Supplied

     |- Cluster Peering Network

     |- ISLs*

#####|Cluster A Site A

     |- 2 x NetApp FAS/AFF Controllers (HA-Pair)

     |- Cluster ISLs

     |- 2 x Fabric Switches

     |- Storage Shelves

     |- Fiber Channel-to-SAS bridges**

#####|Cluster B Site B

     |- 2 x NetApp FAS/AFF Controllers (HA-Pair)

     |- Cluster ISLs

     |- 2 x Fabric Switches

     |- Storage Shelves

     |- Fiber Channel-to-SAS bridges**

* 2 to 4 per fabric

** 2 per stack

Image: Differences between Clustered Data ONTAP MetroCluster Configurations (in 8.3.1)

* Requires Tie Breaker and PVR

Q2: What software versions are we using?

http://mysupport.netapp.com/matrix

http://hwu.netapp.com

CHECK|SOFTWARE

-----+--------

     |ONTAP

     |Fabric Switch Firmware

     |ATTO Bridge Firmware

Download required softwares

- http://mysupport.netapp.com/NOW/cgi-bin/software/

> ONTAP

> Fibre Channel Switch

> Protocol Bridge (for ATTOs)

Download FC Switch Golden Configuration Image files

Download ‘ATTO QuickNAV utility’ to setup ATTOs

Q3: Have we obtained the Hardware Setup Checklist and Software Setup Checklist?

https://mysupport.netapp.com/documentation/productlibrary/index.html?productID=30022

Select your ONTAP version and look for the:

‘MetroCluster Installation and Configuration Guide’

Q4: Device names?

Image: MetroCluster Naming Conventions

2) Configure the MetroCluster Hardware Components

See the ‘MetroCluster Installation and Configuration Guide’

2.1) Install Components and cable the storage controllers

- Rack the components

- Cable the FC-VI and HBA adapters

- Cable the ISLs between MetroCluster sites

- Cable the Cluster Interconnect

- Cable the Cluster Peering connections

- Cable the HA Interconnect

- Cable the Management connections

- Cable the Data ports

- Review the port connections

2.2) Cable and configure storage

- Set shelf IDs

- Configure ATTOs

ATTO Notes:

- Each ATTO bridge must have a unique name

- Configure data rate

- Must set the bridge connection mode to PTP

- Update Firmware

- FibreBridge Installation and Operation Manual available on the ATTO website

ATTO> sastargets

2.3) Configure the FC switches

- FTP Golden Configuration Image files to correct switch

2.4) Repeat for partner cluster

Image: Switch configuration by the NetApp Golden Configuration Image File (Brocade 6505 and 6510)

3) Configure the MetroCluster Software Components

Note: Systems configured in the factory will have much of the below completed.

3.1) Gather the required information

- Switch information

- Cluster creation information

- Node information

- Logical interfaces (LIFs) and ports for cluster peering

- Time server information

- AutoSupport support tool information

- Service processor information

3.2) Verify the disk assignments

Check in Maintenance Mode.

*> storage show shelf

*> storage show disk

*> disk assign -shelf FC_switch_A_1:6.shelf10 -p 0

*> disk assign -shelf FC_switch_B_1:6.shelf20 -p 1

*> disk show -n

*> disk show

3.3) Verify the high-availability (HA) state

*> ha-config show # Should be mcc for all components in 4-node MC, or mcc-2n for 2-node MC

*> ha-config modify controller mcc

*> ha-config modify chassis mcc

3.4) Configure the clusters

- Power up each node

- Enable AutoSupport

- Configure Node Management

::> storage failover show

::> storage failover modify -node localhost -mode ha # If not configured in HA mode

::> net port show -role cluster

::> set adv

::*> network options switchless-cluster modify -enable true # If required

::*> set adm

Note: Recommendation is to use the ‘System Setup tool’ to create the cluster, but do not create aggregates! Or you can use CLI.

3.5) Implement the MetroCluster configuration

- Configure Intercluster LIFs

- Configure network route

CLU1::> cluster peer create -peer-addrs IP1,IP2

CLU2::> cluster peer create -peer-addrs IP1,IP2

::> cluster peer show -instance

::> cluster peer health show

CLU1::> storage aggregate mirror aggr0_controller_A_1

{and repeat for every root aggregate}

CLU1::> storage aggregate create aggr1_controller_A_1 -diskcount X -node controller_A_1 -mirror true

{and repeat for every node - creating a mirrored data aggregate}

::> storage aggregate show-status -aggregate AGGR_NAME

i) Make sure there are 2 non-root mirrored data aggregates present on each cluster (MCC-4 node)

ii) Make sure ha-config state of controller and chassis is mcc (MCC 4 node)> ha-config show

::> metrocluster configure -node-name {NODE_NAME}

Note: This enables metrocluster on all the nodes in the DR group that contains NODE_NAME.

::> metrocluster show

To implement MetroCluster using a Single Data Aggregate

::> set adv

::*> metrocluster configure -allow-with-one-aggregate-true {NODE_NAME}

::*> set adm

4) Prepare to Monitor and Manage the ONTAP MetroCluster Solution

::> metrocluster check run

::> metrocluster check show

::> metrocluster check config-replication show

::> metrocluster check config-replication show -instance

::> metrocluster check config-replication show-aggregate-eligibility

::> metrocluster check node show

::> metrocluster check aggregate show

::> metrocluster check config-replication capture-in-progress show

::> metrocluster check lif show

Run Config Advisor

Send ASUP before testing (on both clusters):

::> system node autosupport invoke -node * -type all -message "Maint=START"

Verify Local HA Operation:

::> storage failover show

::> storage failover takeover controller_A_2

::> storage failover show

::> storage failover giveback controller_A_2

... and repeat ...

Verify Switchover, Healing, and Switchback:

- Run the switchover command

- Confirm that disaster recovery partners have come on line

- Heal the configuration

- Heal the data aggregates

- Heal the root aggregates

- Perform a switchback

Configure the MetroCluster FC Switches for Health Monitoring:

::> storage switch add -switch-ipaddress X.X.X.X

::> storage switch show

Configure the FC-to-SAS Switches for Health Monitoring:

::> storage bridge add -address Y.Y.Y.Y

::> storage bridge show

Protecting Configuration Backup Files Against Disaster (essential for single-node clusters):

::*> system configuration backup settings modify -destination ftp://Z.Z.Z.Z -username admin

::*> system configuration backup settings show

Note 1: Job schedules on both clusters in a MetroCluster configuration must be identical.

Note 2: Both sites of the MetroCluster should be licensed for the same node-locked features.