Sunday, 8 December 2013

PowerShell Script to Delete Aged Snapshots for NetApp Clustered ONTAP

Or: CDOT Snapshot Maintenance Script v 3.0

This post is an update of an update, an evolution of what was posted here on the 29th of November and here on the 13th of November. Here’s hoping this is 3rd time lucky and the script won’t consume much more of my time (just minor tweaks/fixes as required, which I’ll pass on to this post.) I’m pretty happy with the functionality: vserver/volume inclusion/exclusion lists, exclude volumes whose name contains a string, exclude snapshots whose name contains a string, time-stamped log file, send to email, clever enough not to query volumes that aren’t online (since those volumes cannot be queried for snapshots), clever enough to report on but not try deleting snapshots from read-only mirror volumes (cannot delete snapshots from volumes that aren't read/write), … It’s very almost a “one snapshot maintenance script to rule them all”, just not quite. If I ever develop my function to find the oldest day of retention of a snapshot policy (started here) and can incorporate it, then that will be the day it rules them all!

Image: One snapshot maintenance script to rule them all?
Anyway, there’s tons of comment in the script, and hopefully it is fairly self-explanatory and intuitive, so, enough with my inane ramblings and on with the script (apologies using the tiniest font, no colours, and the formatting's gone a bit awry deeper into the script - blogger’s not really designed for posting of PowerShell code!):

######################################################################
## TITLE: CDOT Snapshot Maintenance Script v 3.0                    ##
##  For ALL or SPECIFC vserver/volumes/snapshots, will delete       ##
##  snapshots over X days. NOTE: Comments section at end of script! ##
######################################################################

#########################################
## SECTION 1: USER CONFIGURABLE INPUTS ##
#########################################

# GENERAL SECTION

$runInReportMode        = "YES" # YES/NO
$clusterIP              = "192.168.168.40"
$loginUsername          = "SnapDeletor"
$snapDaysWorthRetention = 5
$logFilePath            = "C:\scripts\logs\"
$logFilePrefixName      = "YOURDC"
$sendEmail              = "NO" # YES/NO
$emailFrom              = "sanmgmt01@domain.com"
$emailTo                = "email1@domain.com,email2@domain.com"
$smtpserver             = "relay.domain.com"
                       
# CUSTOMIZATION SECTION
                       
$vserverInclusionList   = "" # "vs1","vs3" (only what's included)
$vserverExclusionList   = "" # "vs2","vs4" (everything except what's excluded)
$volumeInclusionList    = "" # @(("vs1"),("vol1","vol3"),("vs3"),...)
$volumeExclusionList    = "" # @(("vs2"),("vol2","vol4"),("vs4"),...)
$skipVolIfNameContains  = "" # "*DAG*","*ORA*",...
$skipSnapIfNameContains = "" # "*snapmirror*","*yearly*",...

###########################
## SECTION 2: THE SCRIPT ##
###########################

# The following section creates the filename for the output report file
$fileTag = $logFilePrefixName + "_" + $snapDaysWorthRetention + "_Days_Snap_Del_"
$dateStamp = (get-date).tostring("yyyyMMddHHmm")
$extension = ".log"
$filename = $filetag + $dateStamp + $extension
$fullPathToFile = $logFilePath + $filename

# Custom Out function echos to console and writes to log file
function COUT{ $args; $args >> $fullPathToFile }
if ($runInReportMode -eq "YES"){COUT "@@ RUNNING IN REPORT MODE @@ `r`n"}

# Connect to the Cluster after reading encrypted_password.txt
Import-Module DataOnTap
"Connecting ... this can take a few minutes!"
$encrypted = Get-Content c:\scripts\encrypted_password.txt | ConvertTo-SecureString
$credential = New-Object System.Management.Automation.PsCredential($loginUsername, $encrypted)
Connect-NcController $clusterIP -credential $credential
COUT "Connected to cluster: $clusterIP `r`n"
COUT "The following snapshots are older than $snapDaysWorthRetention days! "

# Query the Vserver Inclusion List or get Vservers (data SVMs only)
if (!$vserverInclusionList){
   $templateVS = Get-NcVserver -Template
   $templateVS.VserverType = "data"
   $getNcVservers = Get-NcVserver -Query $templateVS
   $vservers =  @()
   foreach ($entry in $getNcVservers){$vservers += $entry.Vserver}
} # if !vserverInclusionList
if ($vserverInclusionList){
   $vservers = $vserverInclusionList
   COUT "Using Vserver INCLUSION List! "
} # if vserverInclusionList

# Cycle through VSERVERS
foreach ($vserver in $vservers){
       $one = 1
      
       # This do loop of 1 iteration allows us to stop processing the Vserver if we want
       :STOP_PROCESSING_SVM do{
      
              # Query the Vserver Exclusion List
              if ($vserverExclusionList){
                     foreach ($excludeSVM in $vserverExclusionList){
                           if ($vserver -eq $excludeSVM){
                                  COUT "Vserver $vserver is excluded in the Vserver EXCLUSION List! "
                                  break STOP_PROCESSING_SVM
                           } # if ($vserver -eq $excludeSVM)
                     } # foreach ($excludeSVM in $vserverExclusionList)
              } # if ($vserverExclusionList)
             
              # Set Vserver to query
              COUT "`r`nConnected to vserver $vserver `r`n"
              $global:CurrentNcController.Vserver = $vserver
             
              # Query the Volume Inclusion List or get Volumes
              if (!$volumeInclusionList){
                     $templateVOLS = Get-NcVol -Template
                     $templateVOLS.Name = ""  
                     $getNcVols = Get-NcVol -Attributes $templateVOLS
                     $volumes = @()
                     foreach ($entry in $getNcVols){$volumes += $entry.Name}
              } # if (!$volumeInclusionList)
              if ($volumeInclusionList){
                     $i = 0; $volumes = @()
                     $volIncListArraySize = $volumeInclusionList.count
                     $halfVolIncListArray = $volIncListArraySize / 2
                     do {
                           if ($vserver -eq $volumeInclusionList[$i*2]){
                                  $volumes = $volumeInclusionList[$i*2+1]
                                  COUT "Vserver $vserver has volumes in the volume INCLUSION List! "
                           } # if ($vserver -eq $volumeInclusionList[$i*2])
                           $i++
                     } until ($i -eq $halfVolIncListArray)
              } # if ($volumeInclusionList)
              if (!$volumes){
                     COUT "Vserver $vserver has no volumes in the volume INCLUSION List! "
                     break STOP_PROCESSING_SVM
              } # if !$volumes
             
              # Query the Volume Exclusion List for this Vserver
              if ($volumeExclusionList){
                     $i = 0; $volsInThisVserverExcluded = @()
                     $volIncListArraySize = $volumeExclusionList.count
                     $halfVolIncListArray = $volIncListArraySize / 2
                     do {
                           if ($vserver -eq $volumeExclusionList[$i*2]){
                                  $volsInThisVserverExcluded = $volumeExclusionList[$i*2+1]
                                  COUT "Vserver $vserver has volumes in the volume EXCLUSION List! "
                           }
                           $i++
                     } until ($i -eq $halfVolIncListArray)
              } # if ($volumeExclusionList)
             
              # Cycle through volumes
              foreach ($volume in $volumes){
             
                     # This do loop of 1 iteration allows us to stop processing this volume if we want
                     :STOP_PROCESSING_VOL do{
                            COUT "`r`nScanning volume: $volume `r`n"

                           # Query the Volume Exclusion List for this Volume
                           if ($volsInThisVserverExcluded) {
                                  foreach ($excludeVol in $volsInThisVserverExcluded) {
                                         if ($volume -eq $excludeVol) {
                                                COUT "Volume $volume is in the volume EXCLUSION list! "
                                                break STOP_PROCESSING_VOL
                                         } # if ($volume -eq $excludeVol)
                                  } # foreach ($excludeVol in $volsInThisVserverExcluded)
                           } # if ($volsInThisVserverExcluded)

                           # Query if volume name contains a string marked for exclusion
                           if ($skipVolIfNameContains) {
                                  foreach ($entry in $skipVolIfNameContains) {
                                         if ($volume -like $entry) {
                                                COUT "Volume $volume is EXCLUDED by part of its name! "
                                                break STOP_PROCESSING_VOL
                                         } # if ($volume -like $entry)
                                  } # foreach ($entry in $skipVolIfNameContains)
                           } # if ($skipVolIfNameContains)

                           # Query if volume is online (if not online we cannot read snapshot info)
                           $volStateTemplate = Get-NcVol -Template
                           $volStateTemplate.State = ""
                           $state = get-ncvol $volume -Attributes $volStateTemplate
                           if ($state.state -ne "online"){
                                  COUT "Volume $volume is NOT ONLINE we cannot query it for snapshots! "
                                  break STOP_PROCESSING_VOL
                           } # if ($state.state -ne "online")
                          
                           # Query if the volume is a LS or DP mirror
                           $mirror = ""
                           $volInfo = get-ncvol $volume
                           if ($volInfo.VolumeMirrorAttributes.IsDataProtectionMirror -eq "True"){$mirror = "True"}
                           if ($volInfo.VolumeMirrorAttributes.IsLoadSharingMirror -eq "True"){$mirror = "true"}
                           if ($mirror){COUT "Volume $volume is a MIRROR volume, we will scan for snapshots but cannot delete any! "}
                          
                           # Get the Snapshots
                           $snapshots = Get-NcSnapshot -Volume $volume | where-object {$_.Created -lt (Get-Date).AddDays(-$snapDaysWorthRetention)}
                          
                           # If there are aged snapshots
                           if ($snapshots){
                          
                                  # Cycle through snapshots
                                  foreach ($snapshot in $snapshots){
                                 
                                         # This do loop of 1 iteration allows us to stop processing the snapshot if we want
                                         :STOP_PROCESSING_SNAPSHOT do{
                                                $snapshotName = $snapshot.name
                                                $snapshotDate = $snapshot.created
                                               
                                                # Query if snapshot name contains a string marked for exclusion
                                                if ($skipSnapIfNameContains) {
                                                       foreach ($entry in $skipSnapIfNameContains){
                                                              if ($snapshotName -like $entry) {
                                                                     COUT "Snapshot $snapshotName is EXCLUDED by part of its name! "
                                                                     break STOP_PROCESSING_SNAPSHOT
                                                              } # if ($snapshotName -like $entry)
                                                       } # foreach ($entry in $skipSnapIfNameContains)
                                                } # if ($skipSnapIfNameContains)
                                               
                                                # REMOVE SNAPSHOTS if not running in report mode and not a mirror!
                                                if ($runInReportMode -eq "NO" -and !$mirror){
                                                       Remove-NcSnapshot -Volume $volume -Snapshot $snapshotName -confirm:$false
                                                } # if $runInReportMode
                                               
                                                # Query for the continued existence of the snapshot
                                                $snapStillThere = Get-NcSnapshot -Volume $volume -Snapshot $snapshotName
                                               
                                                # This if statement operates if the snapshot has been deleted
                                                if (!$snapStillThere) {
                                                       COUT "Deleted $snapshotName from $snapshotDate ! "
                                                } # if !$snapStillThere
                                               
                                                # This if statement operates if the snapshot was not deleted
                                                if ($snapStillThere) {
                                                       COUT "$snapshotName from $snapshotDate @@@@@ WAS NOT DELETED! "
                                                } # if $snapStillThere
                                         } until ($one -eq 1) # STOP_PROCESSING_SNAPSHOT do
                                  } # foreach ($snapshot in $snapshots)
                           } # if ($snapshots)
                          
                           # If there are no aged snapshots
                           if (!$snapshots){
                                  COUT "Volume $volume has no snapshots older than $snapDaysWorthRetention days! "
                           } # if !$snapshots  
                     } until ($one -eq 1) # STOP_PROCESSING_VOL do         
              } # foreach $volume       
       } until ($one -eq 1) # STOP_PROCESSING_SVM do
} # foreach ($vserver in $vservers)
  
COUT "`r`nSnapshot deletion complete." "Report saved as $filename. "

################################################
## SECTION 3: Sending the report as an email! ##
################################################

if ($sendEmail -eq "YES"){
       $subject = $filename  
       # Email message contents taken from the output file
       [string]$message = ""
       $messageRaw = get-content -path $fullPathToFile
       foreach ($line in $messageRaw) {$message = $message + $line + "`r`n"}
       # Send the email
       $smtp=new-object Net.Mail.SmtpClient($smtpServer)
       $smtp.Send($emailFrom, $emailTo, $subject, $message)
} # if $sendEmail

######################
## COMMENTS SECTION ##
######################

<#

General:
########

This script will connect to the specified cluster with the specified username, and delete all snapshots beyond a specified $snapDaysWorthRetention retention.

Save this script as say cdotSnapshotMaintenceScript.ps1, and run after setting up environment specific variables in section 1. All the user variables are specified in "Section 1", and there are various (hopefully self-explanatory) options in there. There should be no need to edit any part of the script outside "Section 1".

There is really only one line that does any damage "Remove-NcSnapshot". The script does not force delete snapshots (the PowerShell option "-IgnoreOwners" flag has not been used.) Also, Remove-NcSnapshot will not delete SnapMirror reference snapshots (unlike 7-Mode).

The CUSTOMIZATION section:
##########################

$vserverInclusionList   = "" # only what's included
~Use vserver inclusion list to INCLUDE only specific vserver
$vserverExclusionList   = "" # everything except what's excluded
~Use vserver exclusion list to EXCLUDE specific vservers (i.e. scan all but these)
$volumeInclusionList    = "" # @(("vs1"),("vol1","vol3"),("vs3"),...)
~The volume inclusion list INCLUDES for the specified vserver(s) the specified volume(s)
$volumeExclusionList    = "" # @(("vs2"),("vol2","vol4"),("vs4"),...)
~The volume exclusion list EXCLUDES for the specified vserver(s) the specified volumes(s)
$skipVolIfNameContains  = "" # "VMware","special",...
~Use this to EXCLUDE volumes if they contain a particular string
$skipSnapIfNameContains = "" # "snapmirror","dontDelete",...
~Use this to EXCLUDE snapshots if they contain a particular string

Pre-requisities:
################

1) The file c:\scripts\encrypted_password.txt must have been created first using set-cred.ps1 (commented below for reference)
2) The Data ONTAP PowerShell Toolkit must be installed
3) The PowerShell execution policy set accordingly (remoteSigned will work)
4) Requires at least PowerShell 2.0

set-cred.ps1 file contents:
###########################

# Creates a text file with encrypted password string
# Note: A blank file c:\scripts\encrypted_password.txt must be created first!
$credential = Get-Credential
$credential.Password | ConvertFrom-SecureString | Set-Content c:\scripts\encrypted_password.txt

Using a specific cluster logon account for SnapDeletor:
#######################################################

For a minimum permission account that can be used to run SnapDeletor, run the following from Clustershell.

::> sec login role create -role SnapDeletor -cmddirname DEFAULT -access readonly
::> sec login role create -role SnapDeletor -cmddirname snapshot -access all
::> sec login role create -role SnapDeletor -cmddirname volume -access all
::> sec login create -username SnapDeletor -application ontapi -authmethod password -role SnapDeletor

#>

No comments:

Post a Comment