Tuesday, 12 November 2019

Creating a Custom ONTAP SVM Role for SMBR Restore Purposes

Following on from the last two posts -
- to productionize the tool, you’ll probably not want to hand out the vsadmin credentials. In this post we create a custom security login role for the purposes of performing the SMBR restore (the custom role is still quite powerful.)
Note: Really, this role applies to any use case where one wants to be able to flexclone from a snapshot and perform tidy up with minimum permissions.

Clustershell Commands to Create the Custom Role and User


security login role create -role smbr_restore -vserver svm1 -cmddirname DEFAULT -access none
security login role create -role smbr_restore -vserver svm1 -cmddirname lun -access readonly
security login role create -role smbr_restore -vserver svm1 -cmddirname "lun mapping create" -access all
security login role create -role smbr_restore -vserver svm1 -cmddirname version -access all
security login role create -role smbr_restore -vserver svm1 -cmddirname volume -access readonly
security login role create -role smbr_restore -vserver svm1 -cmddirname "volume clone" -access all
security login role create -role smbr_restore -vserver svm1 -cmddirname "volume destroy" -access all
security login role create -role smbr_restore -vserver svm1 -cmddirname "volume offline" -access all
security login role create -role smbr_restore -vserver svm1 -cmddirname vserver -access readonly

security login create -user smbr_restore -role smbr_restore -vserver svm1 -application ontapi -authentication-method password


Image: Custom ONTAP SVM Role for SMBR Restore Purposes

Note: ONTAP Version used here is ONTAP 9.3.

Saturday, 2 November 2019

Tool to FlexClone SCE Database Snapshot and Present to SMBR Server - Part 2 of 2: Demo

To demonstrate the use of the FlexClone_DB_SnapShot_to_Server tool, here’s a straightforward demonstration with pictures.

1) Double-click FlexClone_DB_SnapShot_to_Server.BAT to run thetool.
2) Input the SVM password.
3) Specify which database you want to restore from.
4) Choose which volume you want to restore from.

Image: 1-4 Selecting Database and Volume

5) Choose which snapshot you want to restore from.
6) Then the tool goes away and flexclones the volume, and presents the LUN as an R drive to the server, which automatically opens.

Image: 5-6 Choosing which snapshot to restore from

Image: The tools runs ‘Explorer R:’ so the Explorer Window automatically pops up and you can browse to your EDB file.

7) Press ENTER when you are read to start the tidy up (Flexclone destroy).

Image: 7 Tidy up completed!

THE END

Tool to FlexClone SCE Database Snapshot and Present to SMBR Server - Part 1 of 2: The Tool

Prerequisites

1) SMBR is installed
2) ONTAP PowerShell Toolkit is installed
3) The server has iSCSI interfaces
4) An initiator group for the server exists on the relevant SVM
5) iSCSI is setup on the SMBR server
6) Credentials to connect to the SVM


Below is an example of the required setup (for 3,4,5). Here we configure MPIO but only have one iSCSI IP on the server (.75) and one iSCSI LIF on the cluster (.131) This is a one-time setup:


get-windowsfeature -name 'Multipath-IO'
add-windowsfeature -name 'Multipath-IO'
get-netipaddress | where-object {$_.AddressFamily -eq "IPv4"}
set-service -name MSiSCSI -StartupType Automatic
Start-service -name MSiSCSI
hostname

  
‌‌
igroup create -vserver svm1 -igroup iqn.1991-05.com.microsoft:smbr.demo.corp.com -protocol iscsi -ostype windows -initiator iqn.1991-05.com.microsoft:smbr.demo.corp.com

  

New-IscsiTargetPortal -InitiatorPortalAddress 192.168.0.75 -TargetPortalAddress 192.168.0.131
get-iscsitarget | connect-iscsitarget -Ispersistent $True -IsMultipathEnabled $True -InitiatorPortalAddress 192.168.0.75 -TargetPortalAddress 192.168.0.131 
Get-IscsiConnection
new-msdsmsupportedHw -VendorId NETAPP -ProductId "LUN C-Mode"
Update-MPIOClaimedHW -Confirm:$false
restart-computer


Running from a Batch File

Image: FlexClone_DB_SnapShot_to_Server.BAT

You’ll need to put a batch file on the desktop, for the exchange administrator to run the tool. The batch file - which we’ll call FlexClone_DB_SnapShot_to_Server.bat - has this content:

PowerShell.EXE C:\Scripts\FlexClone_DB_SnapShot_to_Server.PS1 -SVMName {SVM Name} -HostIGrp {Host Initiator Group Name} -SVMMgmtLIF {SVM Management LIF} -SVMUser vsadmin

PowerShell Script

This the script:


Param(
  [String]$SVMName,
  [String]$HostIGrp,
  [String]$SVMMgmtLIF,
  [String]$SVMUser
)

Function Wr{
  Param([String]$Echo,[String]$Ink = "WHITE")
  If($Echo){Write-Host $Echo -ForegroundColor $Ink -NoNewLine}
  Else{Write-Host}
}

####################
## INITIALIZATION ##
####################

[Void](Import-Module DataONTAP)
If(!(Get-Module DataONTAP)){Wr "Import-Module DataONTAP failed" RED;Wr;PAUSE;EXIT}

Wr;Wr "+++++ Mount DB Snapshot to the SMBR Server +++++" MAGENTA;Wr;Wr
Wr "SVM Name      : " CYAN;If($SVMName){Wr $SVMName;Wr}else{$SVMName = Read-Host}
Wr "Host IGroup   : " CYAN;If($HostIGrp){Wr $HostIGrp;Wr}else{$HostIGrp = Read-Host}
Wr "SVM Mgmt. LIF : " CYAN;If($SVMMgmtLIF){Wr $SVMMgmtLIF;Wr}else{$SVMMgmtLIF = Read-Host}
Wr "SVM Username  : " CYAN;If($SVMUser){Wr $SVMUser;Wr}else{$SVMuser = Read-Host}
Wr "SVM Password  : " CYAN;$SecurePW = Read-Host -AsSecureString
Wr "Recovery Drive: " CYAN;Wr "R";Wr
$Creds = New-Object System.Management.Automation.PsCredential($SVMUser,$SecurePW)

## CHECKING SVM CONNECTION ##
[Void](Connect-NcController -Name $SVMMgmtLIF -Credential $Creds -Vserver $SVMName -ErrorAction SilentlyContinue)
If(!$Global:CurrentNcController){
        Wr "Connect to $SVMName failed!" RED;Wr;PAUSE;EXIT
};Wr

## CHECKING IGROUP EXISTENCE ##
If(!(Get-NcIgroup | Where{$_.Name -eq $HostIGrp})){Wr "Could not find initiator group $HostIGrp" RED;Wr;PAUSE;EXIT}

## CHECKING ISCSI CONNECTIONS ##
$ConnectedCount = 0
[System.Object]$HostIscsi = Get-iSCSIConnection
ForEach($SANLIF in Get-NcISCSIInterface){
  $HostIscsi | Foreach{
    If($SANLIF.IpAddress -eq $_.TargetAddress){
      Wr (($_.InitiatorAddress) + " is connected to " + ($SANLIF.IpAddress)) GREEN;Wr
      $ConnectedCount++
    }
  }
}
If($ConnectedCount -ge 1){
  Wr "At least one iSCSI connection detected!" GREEN; Wr;
  Wr "Note: Currently, we only check for at least one connection. This does not guarantee the LUN mapping will work!" CYAN;Wr
}Else{
  Wr "No iSCSI connections detected!" RED;Wr;PAUSE;EXIT
};Wr

##########################################################
## CUSTOMER INPUTS - customization will be needed here! ##
##########################################################

[System.Array]$DBs = "1","2","3","4","5"
$GoodInput = $FALSE
While(!$GoodInput){
  Wr "Which database do you want to restore from (1-5)? " CYAN;[String]$Database = Read-Host
  $DBs | Foreach{If($Database -eq $_){$GoodInput = $TRUE}}
}

$Attrs = Get-NcVol -Template
$Attrs.VolumeSnapshotAttributes = @{}
Wr;Wr "Searching for volumes..." MAGENTA;Wr;Wr
[System.Object]$Vols = Get-NcVol -Attributes $Attrs | where{$_.name.contains("DB" + "$Database")}
Wr "Volume Name (snapshot count)" CYAN;Wr
$Vols | Foreach{
  Wr ($_.Name + " (" + $_.VolumeSnapshotAttributes.SnapshotCount + ")");Wr
};Wr

$GoodInput = $FALSE
While(!$GoodInput){
  Wr "Which volume do you want to restore from? " CYAN;[String]$RestoVol = Read-Host
  $Vols | Foreach{
    If($RestoVol -eq $_.Name){$GoodInput = $TRUE}
  }
}

$Attrs = Get-NcSnapshot -Template
$Attrs.AccessTime = ""
Wr;Wr "Searching for snapshots..." MAGENTA;Wr;Wr
[System.Objects]$Snaps = Get-NcSnapshot -Volume $RestoVol -Attributes $Attrs
Wr "Snapshot name' created @ MM/DD/YYYY HH:MM:SS AM/PM" CYAN;Wr
$Snaps | Foreach{
  Wr ($_.Name) YELLOW;Wr " @ " CYAN;Wr ($_.Created);Wr
};Wr

$GoodInput = $FALSE
While(!$GoodInput){
  Wr "Which " CYAN; Wr "snapshot name " YELLOW;Wr " do you want to restore from? " CYAN
  [String]$RestoSnap = Read-Host
  $Snaps | Foreach{
    If($RestoSnap -eq $_.Name){$GoodInput = $TRUE}
  }
};Wr

###########
## CLONE ##
###########

$CloneVolName = ("$RestoVol" + "_CLONE")
Wr "Creating clone volume: " MAGENTA; Wr $CloneVolName YELLOW;Wr
Wr "...from snapshot: " MAGENTA; Wr $RestoSnap YELLOW;Wr
Wr "...of volume: " MAGENTA; Wr $RestoVol YELLOW;Wr
[Void](New-NcVolClone -CloneVolume $CloneVolName -Parentvolume $RestoVol -ParentSnapshot $RestoSnap -VolumeType RW)

## FIRST CHECK TO SEE IF ANYTHING MAPPINGS EXIST ##
$Query = Get-NcLunMap -Template
$Query.InitiatorGroup = $HostIGrp
[System.Object]$LunMaps = Get-NcLunMap -Query $Query
$LUNID = 1
If($LunMaps.count -ne 0){
  Wr "There are already LUN mappings to this server. These may need tidy up. Check with the Storage Team." YELLOW;Wr
  $FoundFreeLUNID = $FALSE
  While(!$FoundFreeLUNID){
    $LunMaps | Foreach{
      If($_.LunID -ne $LUNID){$FoundFreeLUNID = $TRUE}
      Else{$LUNID++}
    }
  }
}

## MAP THE LUN (assuming one LUN per volume) ##
$GetLUN = Get-NcLun -Volume $CloneVolName
$LUNPath = $GetLUN.Path
$LUNSerial = $GetLUN.SerialNumber
[Void](Add-NcLunMap -Path $LUNPath -InitiatorGroup $HostIGrp -Id $LUNID)

## WINDOWS CONFIGURATION (assuming one LUN per volume) ##
Update-HostStorageCache
[System.Object]$Disk = Get-Disk | Where{$_.FriendlyName.contains("NETAPP")}
$Disk | Set-Disk -IsReadonly $FALSE
$Disk | Set-Disk -IsOffline $FALSE
$Disk | Get-Partition | Where{$_.Type -eq "Basic"} | Set-Partition -NewDriveLetter R
Update-HostStorageCache
Explorer R:

## NOW DO THE RESTORE ##
Wr "LUN has been mounted as the R: drive." GREEN;Wr
Wr "Use SMBR to restore your data." CYAN;Wr;Wr
Wr "ONLY PRESS ENTER WHEN READY TO START THE TIDY UP." RED;Wr;Wr
PAUSE

#############
## TIDY UP ##
#############

$Disk | Set-Disk -IsOffline $TRUE
[Void](Set-NcVol -Name $CloneVolName -Offline)
[Void](Remove-NcVol -Name $CloneVolName -Confirm:$False)
Update-HostStorageCache
Wr;Wr "Tidy up completed!" GREEN;Wr;Wr
PAUSE


Friday, 1 November 2019

Custom Schedules for SnapCenter Jobs

A bit of a random post.

Say you find the inbuilt scheduler in SnapCenter is a little bit limiting. For example:

You want to do 22 hourly logs backups at every hour from 0200 to 2300, and 1 daily full & logs backup at 0000. The full & logs backup might take over an hour to run.

- you can’t do that with the inbuilt SnapCenter scheduler, but...

If you use the Windows inbuilt Task Scheduler on the SnapCenter server, and configure OnDemand Snapcenter policies, you can configure the Windows Task Scheduler to kick off OnDemand SnapCenter backups on a schedule!

A proof that it works (have a look at this post for how to setup a scheduled task):

1) We have an Exchange Server Backup Policy set to keep 7 on-demand snapshot copies.
Note: I could set the on-demand retention to say 7 days, but this is a lab and it’s easy and quick to run 8 on-demand backups jobs and see if the retention is indeed 7, rather than having to wait 8 days to prove a point.

Image: Exchange Server Backup Policy

2) We have a Resource Group with the on-demand policy.

Image: Resource group for DB5 on MB3 with ON_DEMAND policy

3) To prove it works, we on-demand backup the resource group 8 times (waiting for ‘Status = Completed’ each time).


PS> New-SMBackup -ResourceGroupName DB5_MB3_ONDEMAND -Policy ON_DEMAND | FT Name,Id,StartTime
PS> New-SMBackup -ResourceGroupName DB5_MB3_ONDEMAND -Policy ON_DEMAND | FT Name,Id,StartTime
PS> New-SMBackup -ResourceGroupName DB5_MB3_ONDEMAND -Policy ON_DEMAND | FT Name,Id,StartTime
PS> New-SMBackup -ResourceGroupName DB5_MB3_ONDEMAND -Policy ON_DEMAND | FT Name,Id,StartTime
PS> New-SMBackup -ResourceGroupName DB5_MB3_ONDEMAND -Policy ON_DEMAND | FT Name,Id,StartTime
PS> New-SMBackup -ResourceGroupName DB5_MB3_ONDEMAND -Policy ON_DEMAND | FT Name,Id,StartTime
PS> New-SMBackup -ResourceGroupName DB5_MB3_ONDEMAND -Policy ON_DEMAND | FT Name,Id,StartTime
PS> New-SMBackup -ResourceGroupName DB5_MB3_ONDEMAND -Policy ON_DEMAND | FT Name,Id,StartTime

Name                                           Id StartTime
----                                           -- ---------
Backup of Resource Group 'DB5_MB3_ONDEMAND'... 2319 11/1/2019 2:29:35 PM

PS> (2310..2319 | Foreach{Get-SmJobSummaryReport -JobId $_}) | FT JobEndDateTime,JobName,Status -Autosize

JobEndDateTime       JobName                                        Status
--------------       -------                                        ------
11/1/2019 2:15:31 PM Create Policy 'ON_DEMAND'                      Completed
11/1/2019 2:17:00 PM Create Resource Group 'DB5_MB3_ONDEMAND'       Completed
11/1/2019 2:20:35 PM Backup of Resource Group 'DB5_MB3_ONDEMAND'... Completed
11/1/2019 2:21:15 PM Backup of Resource Group 'DB5_MB3_ONDEMAND'... Completed
11/1/2019 2:23:02 PM Backup of Resource Group 'DB5_MB3_ONDEMAND'... Completed
11/1/2019 2:24:12 PM Backup of Resource Group 'DB5_MB3_ONDEMAND'... Completed
11/1/2019 2:24:54 PM Backup of Resource Group 'DB5_MB3_ONDEMAND'... Completed
11/1/2019 2:25:43 PM Backup of Resource Group 'DB5_MB3_ONDEMAND'... Completed
11/1/2019 2:26:28 PM Backup of Resource Group 'DB5_MB3_ONDEMAND'... Completed
11/1/2019 2:30:06 PM Backup of Resource Group 'DB5_MB3_ONDEMAND'... Completed


4) And check we only have 7 snaphots as specified by our retention.

cluster2::> snapshot show -volume *DB5* -snapshot *ONDEMAND* -fields snapshot
vserver volume       snapshot
------- ------------ ---------------------------------------------
svm2    vol_N3DB4DB5 DB5_MB3_ONDEMAND_mb3_11-01-2019_14.20.57.5337
svm2    vol_N3DB4DB5 DB5_MB3_ONDEMAND_mb3_11-01-2019_14.22.40.9871
svm2    vol_N3DB4DB5 DB5_MB3_ONDEMAND_mb3_11-01-2019_14.23.48.7528
svm2    vol_N3DB4DB5 DB5_MB3_ONDEMAND_mb3_11-01-2019_14.24.35.4717
svm2    vol_N3DB4DB5 DB5_MB3_ONDEMAND_mb3_11-01-2019_14.25.15.7686
svm2    vol_N3DB4DB5 DB5_MB3_ONDEMAND_mb3_11-01-2019_14.26.10.6750
svm2    vol_N3DB4DB5 DB5_MB3_ONDEMAND_mb3_11-01-2019_14.29.35.8160
7 entries were displayed.


Idea proved!

Next steps: To write a little PowerShell script that you can run from a command prompt/scheduled task - that does what you want - test, and setup the Windows Scheduled Task as per requirements.

Thursday, 31 October 2019

Scheduled Snapshot Purge Script - Part 2 of 2) Setting up a Scheduled Task

In order to setup the scheduled task.

1) Create Basic Task Wizard: Create a Basic Task
Name = Snapshot Purge
NEXT>

Image: Create Basic Task Wizard: Create a Basic Task

2) Create Basic Task Wizard: Task Trigger
When do you want the task to start? Daily
NEXT>

Image: Create Basic Task Wizard: Task Trigger

3) Create Basic Task Wizard: Daily
Choose when you want the task to start and recur.
NEXT>

Image: Create Basic Task Wizard

4) Create Basic Task Wizard: Action
What action do you want the task to perform? Start a program

Image: Create Basic Task Wizard: Action

5) Create Basic Task Wizard: Start a Program
Program/script: powershell.exe
Add arguments (optional): -Noninteractive -Noprofile -Command "&C:\scripts\snapshotdeletor.ps1"
Start in (optional): C:\scripts
NEXT>

Note: To run the script from command prompt you’d use>
powershell.exe -Noninteractive -Noprofile -Command "&C:\scripts\snapshotdeletor.ps1"

Image: Create Basic Task Wizard: Start a Program

6) Create Basic Task Wizard: Summary
Tick ‘Open the Properties dialog for this task when I click Finish’
Click FINISH

Image: Create Basic Task Wizard: Summary

7) Snapshot Purge Properties (Local Computer)
Be sure to select:
Run whether user is logged on or not
Run with highest privileges
Click OK

And enter the password for the account running the task.
Click OK

Image: Snapshot Purge Properties (Local Computer)

THE END