Monday, 2 December 2013

A Difference in Clustered ONTAP v 7-Mode in Deleting SnapMirror Snapshots: Part I

Clustered ONTAP has the intelligence to not let you delete reference (shared) SnapMirror snapshots

What Happens in 7-Mode when we try to delete the Reference SnapMirror Snapshot?

In Data ONTAP 7-Mode, it is possible for a storage admin to delete the only reference SnapMirror snapshot in a SnapMirror relationship. This can be a big problem! When there are no in-common snapshots shared by the source and destination volumes of the SnapMirror relationship, the only recourse is to re-baseline the SnapMirror relationship. If the re-baseline has to be done across a WAN, this can take a long time…

Example:

We have a SnapMirror relationship:

na1> snapmirror status
Snapmirror is on.
Source           Destination      State          Lag        Status
na1:prodVol01    na1:drVol01      Snapmirrored   00:00:16   Idle

These are our SnapMirror snapshots:

na1> snap list

Volume prodVol01
  %/used       %/total  date          name
----------  ----------  ------------  --------
 33% (33%)    0% ( 0%)  Dec 02 17:22  na1(4079432748)_drVol01.1 (snapmirror)

Volume drVol01
  %/used       %/total  date          name
----------  ----------  ------------  --------
  0% ( 0%)    0% ( 0%)  Dec 02 17:22  na1(4079432748)_drVol01.1

We try to delete the destination SnapMirror snapshot and it errors - very good:

na1> snap delete drVol01 na1(4079432748)_drVol01.1
Snapshot operation not allowed on a read-only volume.

We then try to delete the source SnapMirror snapshot and it gets deleted - not good:

na1> snap delete prodVol01 na1(4079432748)_drVol01.1
Snapshot copy na1(4079432748)_drVol01.1 on volume prodVol01 was deleted by the Data ONTAP function snapcmd_delete.

We verify the snapshots to see that indeed the one reference SnapMirror snapshot on prodVol01 has been deleted.

na1> snap list

Volume prodVol01
No snapshots exist.

Volume drVol01
  %/used       %/total  date          name
----------  ----------  ------------  --------
  0% ( 0%)    0% ( 0%)  Dec 02 17:22  na1(4079432748)_drVol01.1

We check status of the SnapMirror:

na1> snapmirror status
Snapmirror is on.
Source                Destination           State          Lag        Status
na1:prodVol01         na1:drVol01           Snapmirrored   00:02:33   Idle

It still looks okay so thet’s try and update it:

na1> snapmirror update -S na1:prodVol01 na1:drVol01
replication.dst.err:error: SnapMirror: destination transfer from na1:prodVol01 to drVol01: incremental update not possible; a resync or initialize is necessary. snapmirror.src.connDropped:error: Error reading/writing to network, connection dropped. Transfer aborted: incremental update not possible; a resync or initialize is necessary.

Oh dear, we cannot update the SnapMirror relationship. Thet’s try a resync:

na1> snapmirror resync -S na1:prodVol01 na1:drVol01
Snapmirror resynchronization of drVol01 to na1:prodVol01: no common snapshot to use as the base for resynchronization: aborting resync. snapmirror.src.resync.snapNotFound:error: Could not find base snapshot to resync volume na1:drVol01 to prodVol01. replication.dst.resync.failed:error: SnapMirror resync of drVol01 to na1:prodVol01: no common snapshot to use as the base for resynchronization.

We must recreate (reinitialize the SnapMirror relationship) and wait for a new baseline copy to complete!

What Happens in Clustered ONTAP when we try to delete the Reference SnapMirror Snapshot?

We have a SnapMirror relationship as below:

na81::> snapmirror show
Source              Destination  Mirror        Relationship
Path          Type  Path         State         Status        Healthy
------------- ---- ------------ ------------- -------------- -------
na81://vs1/testshare
              DP   na81://vs2/testDR
                                Snapmirrored  Idle           true

The SnapMirror snapshots on the source and destination volumes are listed below:

na81::> snapshot show -volume testshare
Vserver  Volume  Snapshot                        State
-------- ------- ------------------------------- --------
vs1      testshare
                 snapmirror.15991611-422d-11e3-8578-123478563412_9_2147484694.2013-11-30_142009
                                                 valid

na81::> snapshot show -volume testDR
Vserver  Volume  Snapshot                        State
-------- ------- ------------------------------- --------
vs2      testDR
                 snapmirror.15991611-422d-11e3-8578-123478563412_9_2147484694.2013-11-30_142009
                                                 valid

We try to delete snapshots on the SnapMirror destination volume and it errors just like 7-Mode - which is good:

na81::> snapshot delete -volume testDR -snapshot snapmirror. 15991611-422d-11e3-8578-123478563412_9_2147484694.2013-11-30_142009 -vserver vs2
Error: command failed on vserver "vs2" volume "testDR" snapshot "snapmirror.15991611-422d-11e3-8578-123478563412_9_2147484694.2013-11-30_142009": Snapshot copies can only be deleted on Read_Write (RW) volumes.

We try to delete the snapshot on the SnapMirror source volume and it errors (unlike 7-Mode) - this is excellent! Storage admins and support can rejoice - no longer is it possible to accidentally delete a SnapMirror reference snapshot!

na81::> snapshot delete -volume testshare -vserver vs1 -snapshot snapmirror.15991611-422d-11e3-8578-123478563412_9_2147484694.2013-11-30_142009
Error: command failed: This Snapshot copy is currently used as a reference Snapshot copy by one or more SnapMirror relationships. Deleting the Snapshot copy can cause future SnapMirror operations to fail.

It similarly fails to delete when using the Data ONTAP PowerShell Toolkit:

PS C:\PowerShell> Remove-NcSnapshot -volume testshare -vserver vs1 -Snapshot snapmirror.15991611-422d-11e3-8578-123478563412_9_2147484694.2013-11-30_142009
Delete snapshot: Are you sure you want to delete snapshot snapmirror.15991611-422d-11e3-8578123478563412_9_2147484694.2013-11-30_142009 from volume testshare? YES
Remove-NcSnapshot: This Snapshot copy is currently used as a reference Snapshot copy by one or more SnapMirror relationships. Deleting the Snapshot copy can cause future SnapMirror operations to fail. InvalidOperation:  EOPNOTSUPPORTED

Image: Rejoice - no more accidentally deleting SnapMirror reference snapshots!

To be continued in a Part II ...

For Reference: To Force Delete the SnapMirror Snapshot

na81::> set diag
na81::*> snapshot delete -vserver vs1 -volume testshare -snapshot snapmirror.15991611-422d-11e3-8578-123478563412_9_2147484694.2013-11-30_142009 -ignore-owners true
Warning: This Snapshot copy is currently used as a reference Snapshot copy by one or more SnapMirror relationships. Deleting the Snapshot copy can cause future SnapMirror operations to fail. Are you sure you want to delete 'snapmirror.15991611-422d-11e3-8578-123478563412_9_2147484694.2013-11-30_142009' for volume 'testshare' in Vserver 'vs1'? YES

1 comment:

  1. Thanks, was looking for the force option in cDOT...

    ReplyDelete