Sunday, 25 September 2016

Comparing Aggregate Options Across All Aggregates in a Cluster

The following script will collect all the aggregate options of all the aggregates in a cluster, and output into a CSV, so you can compare. Simply run in PowerShell as PS>


.\Compare-AggrOptions.ps1


An example of the CSV output is below:

Image: AggrOptions.CSV Output

The Script

Copy into a text editor and save as Compare-AggrOptions.ps1.


Param(
  [Parameter(Mandatory=$true)][String]$Cluster,
  [String]$ClusterUserName,
  [String]$ClusterPassword
)

## GENERIC: DISPLAY FUNCTION
Function Wr{Param([String]$Echo = "",[String]$Ink = "WHITE"); Write-Host $Echo -ForegroundColor $Ink}

## GENERIC: LOAD THE DATA ONTAP PSTK
If(!(Get-Module DataONTAP)){ [Void](Import-Module DataONTAP -ErrorAction SilentlyContinue) }
If(Get-Module DataONTAP){ Wr "Loaded DataONTAP PSTK" GREEN }
else{ Wr "Failed to load DataONTAP PSTK!" RED; EXIT }

## GENERIC: CONNECT TO CLUSTER
If($ClusterUserName -and $ClusterPassword){
  $SecPass = $ClusterPassword | ConvertTo-SecureString -asPlainText -Force
  $Cred = New-Object System.Management.Automation.PsCredential($ClusterUserName,$SecPass)
  [Void](Connect-NcController $Cluster -Credential $Cred -ErrorAction SilentlyContinue)
} else {
  If(!(Get-NcCredential $Cluster)){
    $Cred = New-Object System.Management.Automation.PsCredential($(Read-Host "ClusterUserName"),$(Read-Host "ClusterPassword" -AsSecureString))
    [Void](Add-NcCredential -Controller $Cluster -Credential $Cred)
  }
  [Void](Connect-NcController $Cluster -ErrorAction SilentlyContinue)
}
If($Global:CurrentNcController){ Wr "Connected to $Cluster" GREEN}
else{ Wr "Failed to connnect to $Cluster!" RED; EXIT }

## CREATE CSV OF AGGREGATES AND AGGREGATE OPTIONS
$AggrAttr = get-ncaggr -Template
[System.Array]$Aggrs = (Get-NcAggr -Attributes $AggrAttr).Name
$OptionNames = (Get-NcAggrOption $Aggrs[0]).Name

[System.Array]$CSV = @()
$Aggrs | Foreach {
  $aggrOpts = Get-NcAggrOption $_
  $O = New-Object PSObject
  Add-Member -InputObject $O -MemberType NoteProperty -Name "Aggregate" -Value $_
  Foreach ($Option in $OptionNames){
    [String]$Value = ($aggrOpts | Where{$_.Name -eq $Option}).Value
    Add-Member -InputObject $O -MemberType NoteProperty -Name $Option -Value $Value
  }
  $CSV += $O
}

$CSV | Where-Object {$_} | Export-CSV -Path "AggrOptions.CSV" -NoTypeInformation


Note: This was tested against CDOT 8.3.2.

Tuesday, 13 September 2016

Customizing 7MTT Transitions with 7MTT 2.3.1 or 3.0

Note: For to-be-unexplained reasons, I’m looking at just 7MTT 2.3.1 and 7MTT 3.0 for this post…

“Configuration Panning enables you to customize the way 7-Mode Transition Tool transitions the configurations.”

7MTT GUI

In the GUI, you’ll come to the following screen (looks pretty much the same in 7MTT 2.3.1 and 3.0):

Image: 7MTT Configuration Planning customization
SVM Configuration Customizations

CIFS: CIFS Options
CIFS: CIFS User Mapping Configuration
CIFS: Widelinks
CIFS: CIFS Local Users and Groups
CIFS: Preferred Domain Controllers List
CIFS: Audit Configuration
NFS: NFS Options
Name Services: UNIX Users and Groups
Name Services: Name Service Switch Configuration (/etc/nsswitch.conf)
Name Services: Hosts Configuration (/etc/hosts)
Name Services: Netgroups
Name Services: NIS Configuration
Name Services: DNS Configuration
Name Services: LDAP Configuration
Name Services: Name Mapping Switch Configuration

Volume Configuration Customizations

CIFS: Shares and ACLs
CIFS: Home Directory Paths
CIFS: Symlinks
NFS: NFS Exports
NFS: Consolidate 7-Mode NFS Export Rules
NFS: Reuse Export Policies of SVM
SAN: Igroups and LUN Mapping
Snapshot Schedules: Snapshot Schedules
Snapshot Schedules: Reuse Snapshot Policies of SVM
Snapshot Schedules: Consolidate 7-Mode Snapshot Policies

7MTT 2.3.1 CLI

7MTT 2.3.1 CLI uses the> transition property-set


transition property-set -s SESSION_NAME -p PROPERTY_NAME -v PROPERTY_VALUE

#############
## GENERIC ##
#############

-p target-volume-mount-policy
-v preserve_7mode_mount_path (default)
-v 7mode_volume_name
-v target_volume_name

##########
## CIFS ##
##########

-p cluster-audit-saveas-path
-v - (default)
-v /vol//directory/..
-v /vol/

-p ignore-local-users-groups-transition
-v false (default) {or} true

-p ignore-cifs-home-directory-paths-transition
-v false (default) {or} true

-p ignore-cifs-symlinks-transition
-v false (default) {or} true

-p ignore-cifs-widelinks-transition
-v false (default) {or} true

-p ignore-cifs-shares-and-acls-transition
-v false (default) {or} true

-p ignore-cifs-options-transition
-v false (default) {or} true

-p ignore-cifs-name-mapping-transition
-v false (default) {or} true

-p ignore-cifs-audit-transition
-v false (default) {or} true

-p ignore-cifs-preferred-domain-controllers-list-transition
-v false (default) {or} true

-p ignore-all-cifs-configurations-transition
-v true {or} false

#########
## SAN ##
#########

-p ignore-igroup-and-lunmapping-transition
-v false (default) {or} true

-p ignore-all-san-configurations-transition
-v true/false

###################
## NAME SERVICES ##
###################

-p ignore-netgroups-transition
-v false (default) {or} true

-p ignore-unix-users-groups-transition
-v false (default) {or} true

-p ignore-nis-transition
-v false (default) {or} true

-p ignore-dns-transition
-v false (default) {or} true

-p ignore-ldap-transition
-v false (default) {or} true

-p ignore-nsswitch-transition
-v false (default) {or} true

-p ignore-nmswitch-transition
-v false (default) {or} true

-p ignore-etc-hosts-transition
-v false (default) {or} true

-p ignore-all-nameservices-configurations-transition
-v true/false

#########
## NFS ##
#########

-p ignore-nfs-exports-transition
-v false (default) {or} true

-p ignore-nfs-options-transition
-v false (default) {or} true

-p ignore-all-nfs-configurations-transition
-v true/false

-p nfs-consolidate-similiar-7mode-exports
-v true (default) {or} false

-p nfs-reuse-matching-svm-export-policies
-v true (default) {or} false

########################
## SNAPSHOT SCHEDULES ##
########################

-p ignore-snapshot-schedule-transition
-v false (default) {or} true

-p consolidate-similar-7mode-snapshot-policies
-v true (default) {or} false

-p reuse-matching-svm-snapshot-policies
-v true (default) {or} false


7MTT 3.0 CLI

7MTT 3.0 CLI uses the> transition cbt property-set

And apart from -s SESSION_NAME changing to -p PROJECT_NAME, and -p PROPERTY_NAME changing to -n PROPERTY_NAME, the options are identical.


transition cbt property-set -p PROJECT_NAME -n PROPERTY_NAME -v PROPERTY_VALUE


Monday, 12 September 2016

RAID Groups Calculator (RAID-DP) – Revisited

An update of this post. An example of it in action is below. It’s very simple to run, just run as .\RGcalc.ps1 the first time and it will tell you the command line syntax if you want to re-run and make minor changes.

Image: RGcalc.ps1 in action!

The Script

Copy the below into to a text editor and save as RGcalc.ps1:


##########################
## RGcalc.ps1 (RAID-DP) ##
##########################

Param(
  [Parameter(Mandatory=$True,HelpMessage="Number of shelves on the node/HA-pair.")]
  [alias("SC")][Int]$Shelf_Count___,
  [Parameter(Mandatory=$True,HelpMessage="Number of disks in a shelf (12/14/24/48).")]
  [alias("SD")][Int]$Disks_In_Shelf,
  [Parameter(Mandatory=$True,HelpMessage="If you have require root disks (3 for a node/6 for a HA-pair).")]
  [alias("RD")][Int]$Root_Disks____,
  [Parameter(Mandatory=$True,HelpMessage="Number of spares on the node/HA-pair.")]
  [alias("SP")][Int]$Spares_Count__,
  [Parameter(Mandatory=$True,HelpMessage="Number of other disks to be removed from the calculation.")]
  [alias("OD")][Int]$Other_Disks___,
  [Parameter(Mandatory=$True,HelpMessage="Target RAID group size.`nTYPE: Default => Maximum`nSSD: 23 => 28`nSAS/FC: 16 => 28`nSATA: 14 => 20`n6TB MSATA: 14 => 14")]
  [alias("RG")][Int]$Target_RG_Size,
  [Parameter(Mandatory=$True,HelpMessage="Answer 'y' or 'n' depending on if you want a fixed RAID group size or not.")]
  [alias("Fx")][String]$Fixed_RG_Size_,
  [Parameter(Mandatory=$True,HelpMessage="Right-Size info to get aggr capacity.")]
  [alias("RS")][Int]$Disk_RightSize
)

## ===== PARAM VALIDATION ===== ##
If("y","n" -NotContains $Fixed_RG_Size_){"Answer y or n for FixedRGsize!";EXIT}
$Shelf_Count___,$Disks_In_Shelf,$Target_RG_Size | Foreach{If($_ -le 1){"Invalid input!";EXIT}}
$Root_Disks____,$Spares_Count__,$Other_Disks___,$Disk_RightSize | Foreach{If($_ -lt 0){"Invalid input!";EXIT}}

## ===== CALCULATION ===== ##
Function RoundUp{Param([Double]$N);[Int][Math]::Ceiling($N)}
[Int]$AvailableDisks = ($Shelf_Count___ * $Disks_In_Shelf) - $Root_Disks____ - $Spares_Count__ - $Other_Disks___
[Int]$RaidGroups = RoundUp($AvailableDisks/$Target_RG_Size)
[Int]$AvrgRGsize = RoundUp($AvailableDisks/$RaidGroups)
[Int]$LastRGsize = $AvailableDisks - (($RaidGroups -1)*$AvrgRGsize)
[Int]$Balancer = RoundUp(($AvrgRGsize - $LastRGsize) / $RaidGroups)
[System.Array]$Output = @()
For($i=1;$i -lt $RaidGroups;$i++){
  If(($LastRGsize + $Balancer) -lt $Target_RG_Size){
    $Output += ($AvrgRGsize - $Balancer)
    $LastRGsize += $Balancer
  }else{ $Output += $AvrgRGsize }
}
$Output += $LastRGsize
$Output = $Output | Sort -Descending

## ===== OVERRIDE FOR FIXED RG SIZE ===== ##
If($Fixed_RG_Size_ -eq "y"){
  If(($Target_RG_Size * $RaidGroups) -gt $AvailableDisks){ $RaidGroups-- }
  $Spares_Count__ = $Spares_Count__ + $AvailableDisks - ($RaidGroups * $Target_RG_Size)
  $AvailableDisks = ($RaidGroups * $Target_RG_Size) # Available disks for RAID Groups needs recalculation with -FixedRGsize
  [System.Array]$Output = @()
  For($j=1;$j -lt $RaidGroups;$j++){ $Output += $Target_RG_Size }
};""

## ===== OUTPUT ===== ##
">>>>> RAID Groups Calculator (RAID-DP) - RGcalc.ps1 <<<<<";""
"RAID Group: Count = " + [String]$RaidGroups
"RAID Group: Sizes = " + [String]$Output;""
[Int]$DataDisks = $AvailableDisks - ($RaidGroups * 2)
"DISKS: Data   = " + [String]$DataDisks
"DISKS: Parity = " + [String]($RaidGroups * 2)
"DISKS: Spares = " + [String]$Spares_Count__
"DISKS: Other  = " + [String]($Root_Disks____ + $Other_Disks___)
"------------- -"
"DISKS: TOTAL  = " + [String]($Shelf_Count___ * $Disks_In_Shelf);""
If($Disk_RightSize -ne 0){
  [Int]$AggrCap = $DataDisks * $Disk_RightSize
  "100%/97%/90% AGGR CAPACITY = " + [String]$AggrCap + " / " + [String]([Int]($AggrCap*0.97)) + " / " + [String]([Int]($AggrCap*0.90));""
}
"PS> .\RGcalc.ps1 -SC $Shelf_Count___ -SD $Disks_In_Shelf -RD $Root_Disks____ -SP $Spares_Count__ -OD $Other_Disks___ -RG $Target_RG_Size -Fx $Fixed_RG_Size_ -RS $Disk_RightSize";""


Monday, 5 September 2016

Pre-Stage SVM-DR Relation pre 7 to C Cutover

If you’ve got big volumes to transition from 7 to C, and want SVM-DR functionality almost on the button post 7 to C cutover (i.e. it’s not acceptable to have to wait for the entire SVM to SnapMirror across to DR), it’s possible to pre-stage the SVM-DR relation. Here’s an example to show how:

Note: cDOT version here is 8.3.2. 7MTT version is 2.3.1.

1) Create the initial baseline data copy from 7-Mode to cDOT.

Image: 7-Mode volume baselined to cDOT

1b) Extra Belt and Braces Step

Create a snapshot on the 7-Mode source volume(s) "snap create VOLNAME SNAPNAME", and perform an update with 7MTT. The success of the SVM-DR resync depends upon having an in-common 7-Mode snapshot.

2) Stop the 7MTT service on the 7MTT box:

Via DOS/PowerShell>

net stop "Transition Tool"

3) On source cluster (clu1), we break the 7 to C SnapMirror, and remove volume transition protect:


clu1::>

snapmirror break -destination-path FAS101_clu1:TVOL001
set -c off; set d
volume transition-protect -vserver FAS101_clu1 -volume TVOL001 -is-enabled off


4) On the DR cluster (clu2), we create the SVM-DR vserver, vserver peer, and then accept the vserver peer on clu1:


clu2::>

vserver create -vserver FAS101_clu2 -subtype dp-destination
vserver peer create -vserver FAS101_clu2 -peer-cluster clu1 -peer-vserver FAS101_clu1 -applications snapmirror

clu1::>

vserver peer accept -vserver FAS101_clu1 -peer-vserver FAS101_clu2


5) We create the SVM-DR relation, initialize, and wait for initialization to complete:


clu2::>

snapmirror create -source-path FAS101_clu1: -destination-path FAS101_clu2: -identity-preserve true
snapmirror initialize -destination-path FAS101_clu2:
snapmirror show -destination-path FAS101_clu2:


6) Once the SVM-DR relation is initialized, we break and delete it:


clu2::>

snapmirror break -destination-path FAS101_clu2:
snapmirror delete -destination-path FAS101_clu2:


7) Then we release the SVM-DR relation on the source cluster, and resync the 7 to C SnapMirror:


clu1::>

snapmirror release -destination-path FAS101_clu2:TVOL001
snapmirror resync -destination-path FAS101_clu1:TVOL001


Note: You have to do the delete and release, otherwise the 7 to C resync errors with "Destination ... cannot be the source of another SnapMirror relationship"

8) Restart the 7MTT service on the 7MTT box:

Via DOS/PowerShell>

net start "Transition Tool"

And add in the credentials to the 7-Mode Controllers and ONTAP systems.

9) {When the time comes...*} Complete the Transition via 7MTT!

Image: Completed 7 to C Transition

*You don’t want this to be too long after you’ve seeded SVM-DR since the in-common Snapshots may be lost.

10) Use sync-cron.ps1 from the following link to ensure cron schedules are in sync from source to DR clusters:


11) Re-create the SVM-DR relation and resync:


clu2::>

snapmirror create -source-path FAS101_clu1: -destination-path FAS101_clu2: -identity-preserve true
snapmirror resync -destination-path FAS101_clu2:


IMPORTANT: If you are considering using this method, please test thoroughly in your environment first.

APPENDIX: Event wafl.snaprestore.revert

Of course, for the SVM-DR resync to work, there’s got to be in-common snapshots (otherwise it’s a full baseline resync again.) You might see events like the below when the volume SnapRestore reverts to the common (previous) Snapshot copy:


clu2::> event log show -event wafl.snaprestore.revert* -instance

Node: clu2n1
Sequence#: 11211
Time: 9/6/2016 00:16:02
Severity: INFORMATIONAL
Source: config_async_0
Message Name: wafl.snaprestore.revert
Event: wafl.snaprestore.revert: Reverting volume TVOL001@vserver:8d16297e-7364-11e6-b6c8-000c29a5823e to a previous Snapshot copy with snapid 1.
Corrective Action: (NONE)
Description: This message occurs during volume initialization when the system reverts to a previous Snapshot(TM) copy.


Installing 7-Mode Shelf, ACP, and Disk Firmware, and Qual_Devices, whilst in the Datacenter

It’s a bit funny to post some 7-Mode stuff, but it’s still out there, and finally I learnt how it’s really easy to do the titular task!

Scenario

We’re in the datacentre, we’ve just got our trusty laptop, Mongoose Web Server, a console cable, an Ethernet cable, and the required softwares, to facilitate installing Shelf, ACP, and Disk Firmware, and Qual_Devices.

Installing ZIPs

Laptop IP is 169.254.254.1/16 (an APIPA IP that no one should be using.) We’re plugging into a spare and unused e0b port on the back of the controller. We first set e0b’s IP address, get and install the software, then clear e0b’s IP address:


ifconfig -a
ifconfig e0b 169.254.254.2 netmask 255.255.0.0
ifconfig e0b

software install http://169.254.254.1:8080/all_shelf_fw.zip -f
software install http://169.254.254.1:8080/all.zip -f
software install http://169.254.254.1:8080/qual_devices.zip -f

ifconfig e0b 0.0.0.0


Note: The -f (for force) just overwrites the files if they already exist in /etc/software.

Installing ZIPs more Slowly and Checking

Same as above, except this time we do get and then install later, and review the contents of the various directories to see they have been updated:


ifconfig -a
ifconfig e0b 169.254.254.2 netmask 255.255.0.0
ifconfig e0b

software get http://169.254.254.1:8080/all_shelf_fw.zip -f
software get http://169.254.254.1:8080/all.zip -f
software get http://169.254.254.1:8080/qual_devices.zip -f

priv set advanced
ls /etc/software

ifconfig e0b 0.0.0.0

ls /etc/shelf_fw
ls /etc/acpp_fw
ls /etc/disk_fw
rdfile /etc/qual_devices_v3

software install all_shelf_fw.zip
software install all.zip
software install qual_devices.zip

ls /etc/shelf_fw
ls /etc/acpp_fw
ls /etc/disk_fw
rdfile /etc/qual_devices_v3

sysconfig -v
storage show acp


Installing TAR and GZ Files

If you’ve not got the ZIP variants, then you can’t use “software install”. Here’s what we do:


ifconfig -a
ifconfig e0b 169.254.254.2 netmask 255.255.0.0
ifconfig e0b

software get http://169.254.254.1:8080/all_shelf_fw.tar -f
software get http://169.254.254.1:8080/all.gz -f
software get http://169.254.254.1:8080/qual_devices_tar.gz -f

priv set diag
ls /etc/software

ifconfig e0b 0.0.0.0

ls /etc/shelf_fw
ls /etc/acpp_fw
ls /etc/disk_fw
rdfile /etc/qual_devices_v3

useradmin diaguser password
useradmin diaguser unlock
systemshell

login: diag
Password:

cd /mroot/etc/software

tar -zxvf all_shelf_fw.tar
tar -zxvf all.gz
tar -zxvf qual_devices_tar.gz

mv /mroot/etc/software/shelf_fw/* /mroot/etc/shelf_fw/
mv /mroot/etc/software/acpp_fw/* /mroot/etc/acpp_fw/
mv /mroot/etc/software/disk_fw/* /mroot/etc/disk_fw/
mv /mroot/etc/software/qual_devices_v3 /mroot/etc/qual_devices_v3

rm /mroot/etc/software/all_shelf_fw.tar
rm /mroot/etc/software/all.gz
rm /mroot/etc/software/qual_devices_tar.gz
ls /mroot/etc/software

exit

storage download shelf
storage download acp
options raid.background_disk_fw_update.enable # Default is on
options raid.background_disk_fw_update.enable on # If off

sysconfig -v
storage show acp

useradmin diaguser lock


THE END