cDOT Datacenter Relocation Experiment: Part 2/2 “The Clustershell”

The following takes us step by step through the entire scenario detailed in Part 1, focusing on one volume called AVOL01. For this lab I’m essentially interested in verifying the replication strategy is sound, hence no attention to data serving protocols here. Note that my Vservers are called CLUSTERNAME-V1. Cluster peers and Vserver peers for the initial setup already exist.

1) Setup

Firstly, we create our test volume - AVOL01, replicate it across the four clusters, and create a few test Snapshots for verification that everything is working as intended.

CLUA1::> vol create -vserver CLUA1-V1 -volume AVOL01 -aggregate N1_aggr1 -size 5g -junction-path /AVOL01 -space-guarantee none -snapshot-policy default
CLUAV::> vol create -vserver CLUAV-V1 -volume AVOL01 -aggregate N1_aggr1 -size 5g -space-guarantee none -snapshot-policy none -type DP
CLUB1::> vol create -vserver CLUB1-V1 -volume AVOL01 -aggregate N1_aggr1 -size 5g -space-guarantee none -snapshot-policy none -type DP
CLUBV::> vol create -vserver CLUBV-V1 -volume AVOL01 -aggregate N1_aggr1 -size 5g -space-guarantee none -snapshot-policy none -type DP

CLUAV::> snapmirror create -source-path CLUA1-V1:AVOL01 -destination-path CLUAV-V1:AVOL01 -type XDP -schedule daily -policy XDPDefault
CLUAV::> snapmirror initialize -destination-path CLUAV-V1:AVOL01
CLUB1::> snapmirror create -source-path CLUA1-V1:AVOL01 -destination-path CLUB1-V1:AVOL01 -type DP -schedule hourly
CLUB1::> snapmirror initialize -destination-path CLUB1-V1:AVOL01
CLUBV::> snapmirror create -source-path CLUAV-V1:AVOL01 -destination-path CLUBV-V1:AVOL01 -type DP -schedule daily
CLUBV::> snapmirror initialize -destination-path CLUBV-V1:AVOL01

CLUA1::> vol snapshot create -vserver CLUA1-V1 -volume AVOL01 -snapshot daily.2015-08-01 -snapmirror-label daily

... create a month worth of dailys ...

CLUA1::> vol snapshot create -vserver CLUA1-V1 -volume AVOL01 -snapshot weekly.2015-08-01 -snapmirror-label weekly

... create a month worth of weeklys ...

CLUB1::> snapmirror update -destination-path CLUB1-V1:AVOL01
CLUAV::> snapmirror update -destination-path CLUAV-V1:AVOL01
CLUBV::> snapmirror update -destination-path CLUBV-V1:AVOL01

CLUB1::> snapshot show -vserver CLUB1-V1 -volume AVOL01
CLUAV::> snapshot show -vserver CLUAV-V1 -volume AVOL01
CLUBV::> snapshot show -vserver CLUBV-V1 -volume AVOL01

2) Replicating to the New Site

Here we create cluster peers and vserver peers to the new site; and setup the SnapMirror relationship.

CLUA1::> cluster peer create -peer-addrs 10.10.10.243
CLUC1::> cluster peer create -peer-addrs 10.10.10.121
CLUA1::> vserver peer create -vserver CLUA1-V1 -peer-vserver CLUC1-V1 -peer-cluster CLUC1 -applications snapmirror
CLUC1::> vserver peer accept -vserver CLUC1-V1 -peer-vserver CLUA1-V1
CLUC1::> vol create -vserver CLUC1-V1 -volume AVOL01 -aggregate N1_aggr1 -size 5g -space-guarantee none -snapshot-policy none -type DP
CLUC1::> snapmirror create -source-path CLUA1-V1:AVOL01 -destination-path CLUC1-V1:AVOL01 -type DP -schedule hourly
CLUC1::> snapmirror initialize -destination-path CLUC1-V1:AVOL01

3) Cutting over to New Primary Site and Removing Original Primary from Peers

Create Cluster and Vserver peers to/from the DR Cluster (CLUB1) and the SnapVault Cluster (CLUAV), to the new Primary Cluster (CLUC1) and its Vserver (CLUC1-V1).

CLUB1::> cluster peer create -peer-addrs 10.10.10.243
CLUC1::> cluster peer create -peer-addrs 10.10.10.221
CLUB1::> vserver peer create -vserver CLUB1-V1 -peer-vserver CLUC1-V1 -peer-cluster CLUC1 -applications snapmirror
CLUC1::> vserver peer accept -vserver CLUC1-V1 -peer-vserver CLUB1-V1

CLUAV::> cluster peer create -peer-addrs 10.10.10.243
CLUC1::> cluster peer create -peer-addrs 10.10.10.126
CLUAV::> vserver peer create -vserver CLUAV-V1 -peer-vserver CLUC1-V1 -peer-cluster CLUC1 -applications snapmirror
CLUC1::> vserver peer accept -vserver CLUC1-V1 -peer-vserver CLUAV-V1

Stop access to data on CLUA1!!!

Update the DR, break and delete.
Update the SnapMirror to the new Primary/Production volume, break and delete.

CLUB1::> snapmirror update -destination-path CLUB1-V1:AVOL01
CLUB1::> snapmirror show -destination-path CLUB1-V1:AVOL01 -fields status,healthy
CLUB1::> snapmirror break  -destination-path CLUB1-V1:AVOL01
CLUB1::> snapmirror delete -destination-path CLUB1-V1:AVOL01

CLUC1::> snapmirror update -destination-path CLUC1-V1:AVOL01
CLUC1::> snapmirror show -destination-path CLUC1-V1:AVOL01 -fields status,healthy
CLUC1::> snapmirror break  -destination-path CLUC1-V1:AVOL01
CLUC1::> snapmirror delete -destination-path CLUC1-V1:AVOL01

Create a new DP Mirror to DR and resync.
Create a new XDP Mirror to the Vault and resync.

CLUB1::> snapmirror create -source-path CLUC1-V1:AVOL01 -destination-path CLUB1-V1:AVOL01 -type DP -schedule hourly
CLUB1::> snapmirror resync -destination-path CLUB1-V1:AVOL01 -type DP
CLUB1::> snapmirror show -destination-path CLUB1-V1:AVOL01 -fields status,healthy

CLUAV::> snapmirror delete -destination-path CLUAV-V1:AVOL01
CLUAV::> snapmirror create -source-path CLUC1-V1:AVOL01 -destination-path CLUAV-V1:AVOL01 -type XDP -schedule daily -policy XDPDefault
CLUAV::> snapmirror resync -destination-path CLUAV-V1:AVOL01
CLUAV::> snapmirror show -destination-path CLUAV-V1:AVOL01 -fields status,healthy

Create a test snapshot on the new primary volume and replicate it to verify the replications are good.

CLUC1::> vol snapshot create -vserver CLUC1-V1 -volume AVOL01 -snapshot daily.TEST -snapmirror-label daily

CLUB1::> snapmirror update -destination-path CLUB1-V1:AVOL01
CLUB1::> snapmirror show -destination-path CLUB1-V1:AVOL01 -fields status,healthy
CLUB1::> vol snapshot show -vserver CLUB1-V1 -volume AVOL01 -snapshot daily.TEST -fields snapshot

CLUAV::> snapmirror update -destination-path CLUAV-V1:AVOL01
CLUAV::> snapmirror show -destination-path CLUAV-V1:AVOL01 -fields status,healthy
CLUAV::> vol snapshot show -vserver CLUAV-V1 -volume AVOL01 -snapshot daily.TEST -fields snapshot

CLUBV::> snapmirror update -destination-path CLUBV-V1:AVOL01
CLUBV::> snapmirror show -destination-path CLUBV-V1:AVOL01 -fields status,healthy
CLUBV::> vol snapshot show -vserver CLUBV-V1 -volume AVOL01 -snapshot daily.TEST -fields snapshot

Release mirrors from the old primary, and delete peers from the old primary.

CLUA1::> snapmirror release -source-path CLUA1-V1:AVOL01 -destination-path CLUAV-V1:AVOL01
CLUA1::> snapmirror release -source-path CLUA1-V1:AVOL01 -destination-path CLUB1-V1:AVOL01
CLUA1::> snapmirror release -source-path CLUA1-V1:AVOL01 -destination-path CLUC1-V1:AVOL01

CLUA1::> vserver peer delete -vserver CLUA1-V1 -peer-vserver CLUAV-V1
CLUA1::> vserver peer delete -vserver CLUA1-V1 -peer-vserver CLUB1-V1
CLUA1::> vserver peer delete -vserver CLUA1-V1 -peer-vserver CLUC1-V1

CLUA1::> cluster peer delete -cluster CLUAV
CLUA1::> cluster peer delete -cluster CLUB1
CLUA1::> cluster peer delete -cluster CLUC1

CLUAV::> cluster peer delete -cluster CLUA1
CLUB1::> cluster peer delete -cluster CLUA1
CLUC1::> cluster peer delete -cluster CLUA1

4) Bringing the SnapVault Across and Renaming the SnapVault Cluster

Since the SnapVault Cluster and Vserver are going to be renamed with the new site naming convention, we must delete and release all mirrors to/from the vault, and delete all peers to/from the vault. Then we rename the cluster and vserver.

CLUAV::> snapmirror delete -destination-path CLUAV-V1:AVOL01
CLUC1::> snapmirror release -source-path CLUC1-V1:AVOL01 -destination-path CLUAV-V1:AVOL01
CLUBV::> snapmirror delete -destination-path CLUBV-V1:AVOL01
CLUAV::> snapmirror release -source-path CLUAV-V1:AVOL01 -destination-path CLUBV-V1:AVOL01

CLUC1::> vserver peer delete -vserver CLUC1-V1 -peer-vserver CLUAV-V1
CLUAV::> vserver peer delete -vserver CLUAV-V1 -peer-vserver CLUBV-V1

CLUC1::> cluster peer delete -cluster CLUAV
CLUAV::> cluster peer delete -cluster CLUC1
CLUAV::> cluster peer delete -cluster CLUBV
CLUBV::> cluster peer delete -cluster CLUAV

CLUAV::> hostname CLUCV

N.B.: The prompt changes from CLUAV to CLUCV

CLUCV::> vserver rename -vserver CLUAV-V1 -newname CLUCV-V1

Create new peers.

CLUC1::> cluster peer create -peer-addrs 10.10.10.126
CLUCV::> cluster peer create -peer-addrs 10.10.10.243
CLUCV::> cluster peer create -peer-addrs 10.10.10.227
CLUBV::> cluster peer create -peer-addrs 10.10.10.126

CLUC1::> vserver peer create -vserver CLUC1-V1 -peer-vserver CLUCV-V1 -peer-cluster CLUCV -applications snapmirror
CLUCV::> vserver peer accept -vserver CLUCV-V1 -peer-vserver CLUC1-V1
CLUCV::> vserver peer create -vserver CLUCV-V1 -peer-vserver CLUBV-V1 -peer-cluster CLUBV -applications snapmirror
CLUBV::> vserver peer accept -vserver CLUBV-V1 -peer-vserver CLUCV-V1

Create new SnapMirrors/SnapVaults.

CLUCV::> snapmirror create -source-path CLUC1-V1:AVOL01 -destination-path CLUCV-V1:AVOL01 -type XDP -schedule daily -policy XDPDefault
CLUCV::> snapmirror resync -destination-path CLUCV-V1:AVOL01
CLUCV::> snapmirror show -destination-path CLUCV-V1:AVOL01 -fields status,healthy

CLUBV::> snapmirror create -source-path CLUCV-V1:AVOL01 -destination-path CLUBV-V1:AVOL01 -type DP -schedule daily
CLUBV::> snapmirror resync -destination-path CLUBV-V1:AVOL01
CLUBV::> snapmirror show -destination-path CLUBV-V1:AVOL01 -fields status,healthy

And test.

CLUC1::> vol snapshot create -vserver CLUC1-V1 -volume AVOL01 -snapshot daily.TEST2 -snapmirror-label daily

CLUB1::> snapmirror update -destination-path CLUB1-V1:AVOL01
CLUB1::> snapmirror show -destination-path CLUB1-V1:AVOL01 -fields status,healthy
CLUB1::> vol snapshot show -vserver CLUB1-V1 -volume AVOL01 -snapshot daily.TEST2 -fields snapshot

CLUCV::> snapmirror update -destination-path CLUCV-V1:AVOL01
CLUCV::> snapmirror show -destination-path CLUCV-V1:AVOL01 -fields status,healthy
CLUCV::> vol snapshot show -vserver CLUCV-V1 -volume AVOL01 -snapshot daily.TEST2 -fields snapshot

CLUBV::> snapmirror update -destination-path CLUBV-V1:AVOL01
CLUBV::> snapmirror show -destination-path CLUBV-V1:AVOL01 -fields status,healthy
CLUBV::> vol snapshot show -vserver CLUBV-V1 -volume AVOL01 -snapshot daily.TEST2 -fields snapshot

Comments