Thursday, 24 March 2016

SVM-DR/SnapMirror for SVM Super (Short) Express Guide (8.3.1)

The following is a super (short) express guide for SVM-DR/SnapMirror for SVM setup, failover, and failback.

*Note that no consideration of Data LIFs, CIFS servers, protocols ... is in this post, it's purely an exercise in understanding how the SnapMirror side of it works. Please read the official guides for the necessary details to get a successful SnapMirror for SVM setup!


1) Establishing SnapMirror for SVM

## Preparation ##

BOTH::> cluster peer show
BOTH::> system license show
BOTH::> job schedule cron show # IMPORTANT: Create the same custom schedules on the destination cluster!
BOTH::> storage aggregate show

## Creating a destination SVM ##

CLU2::> vserver create -vserver SVM1-DR -subtype dp-destination
CLU2::> vserver show -vserver SVM1-DR

## Create the SVM peer relationship ##

CLU2::> vserver peer create -vserver SVM1-DR -peer-vserver SVM1 -applications snapmirror -peer-cluster CLU1
CLU1::> vserver peer accept -vserver SVM1 -peer-vserver SVM1-DR
BOTH::> vserver peer show

## Creating a SnapMirror Relationship ##

CLU2::> snapmirror create -source-vserver SVM1 -destination-vserver SVM1-DR -type DP -throttle unlimited -policy DPDefault -schedule hourly -identity-preserve false
CLU2::> snapmirror show -destination-vserver SVM1-DR
CLU2::> snapmirror initialize -destination-vserver SVM1-DR
CLU2::> snapmirror show -destination-vserver SVM1-DR


And that’s it, SVM-DR configured!*

2) Failing over to the DP-Destination SVM

## Failing over the dp-destination SVM ##

CLU1::> vserver stop -vserver SVM1
CLU1::> vserver show -vserver SVM1

CLU2::> snapmirror update -destination-vserver SVM1-DR
CLU2::> snapmirror show -destination-vserver SVM1-DR
CLU2::> snapmirror quiesce -destination-vserver SVM1-DR
CLU2::> snapmirror show -destination-vserver SVM1-DR
CLU2::> snapmirror break -destination-vserver SVM1-DR
CLU2::> snapmirror show -destination-vserver SVM1-DR
CLU2::> vserver start -vserver SVM1-DR
CLU2::> vserver show -vserver SVM1-DR

# Note: Notice that the dp-destination SVM’s subtype is now default!

3) Failing Back to the Source SVM

## Reactivating the Source SVM ##

CLU1::> snapmirror create -source-vserver SVM1-DR -destination-vserver SVM1 -type DP -throttle unlimited -policy DPDefault -schedule hourly -identity-preserve false
CLU1::> snapmirror show -destination-vserver SVM1
CLU1::> snapmirror resync -destination-vserver SVM1
CLU1::> snapmirror show -destination-vserver SVM1

CLU2::> vserver stop -vserver SVM1-DR
CLU2::> vserver show -vserver SVM1-DR

CLU1::> snapmirror update -destination-vserver SVM1
CLU1::> snapmirror show -destination-vserver SVM1
CLU1::> snapmirror quiesce -destination-vserver SVM1
CLU1::> snapmirror show -destination-vserver SVM1
CLU1::> snapmirror break -destination-vserver SVM1
CLU1::> snapmirror show -destination-vserver SVM1
CLU1::> vserver start -vserver SVM1

## Resynchronizing the destination SVM from the source SVM ##

CLU2::> snapmirror resync -destination-vserver SVM1-DR
CLU2::> snapmirror show -destination-vserver SVM1-DR

## Tidy up ##

CLU1::> snapmirror delete -destination-vserver SVM1
CLU2::> snapmirror release -destination-vserver SVM1


Q: Could an existing pair of SVM’s with existing SnapMirrors be converted into an SVM-DR relation?

This was actually the point of the lab test, to see if it might be possible. And I did try, effectively picking up from the “Failing Back to the Source SVM” workflow. The problem is that SVM-DR detects volumes on the destination, and complains when trying to resync the SVM-DR relation:

Error: command failed: The following volumes "rootvol,vol1,vol2,vol3" on the destination Vserver are new volumes and do not correspond with the volumes on the source Vserver. Delete them using the (privilege: advanced) "volume delete -force" command. When this command completes successfully, try the operation again.

Of course the volumes do correspond, but I guess there’s a database somewhere that says they don’t …


Wednesday, 23 March 2016

7-Mode Config Checks

Just using the blog as a note repository here (it’s often the only place I can find anything… even then, sometimes I totally forget what I’ve put on here…)!

aggr status –v
bmc status
cf monitor all
cf partner
cf status
date
df
df –A
environment status
exportfs
fcp show cfmode
fcstat device_map
hostname
ifconfig –a
igroup show
license
lun show –m
lun show –v
options
rdfile /etc/exports
rdfile /etc/hosts
rdfile /etc/rc
rlm status
sis status
snap list –n
snapmirror status
snapvault status
sp status
storage show acp –a
storage show disk –a
storage show disk –p
sysconfig –r
sysconfig –a
version
vol status –v

Get 7-Mode LUNs Super CSV

Essentially, I just wanted to get the LUN Geometry Size and Max Resize Size, for LUNs being migrated from a 7-Mode system, and then decided, “why not create a LUNs Super CSV” - then I can sort and filter to my heart’s content in Excel. The CSV contains all LUNs and their Get-NaLun, Get-NaLunGeometry, and Get-NaLunMap info; with my bonus column of GeometryMaxGrowthMultiplier (Max Resize Size / Size). Save the script as say Get-NaLUNsSuperCSV.ps1, and run as in the screen shot below:


The Script

########################
## Get-NaLUNsSuperCSV ##
########################

## ===== GENERIC FUNCTIONS ===== ##
Function Wr{
  Param([String]$Echo,[String]$Ink = "WHITE")
  If($Echo){ Write-Host $Echo -ForegroundColor $Ink -NoNewLine }else{ Write-Host }
}
Function Get-Prompt{
  While($TRUE){
    Wr $Args CYAN; $ReadHost = Read-Host
    If($ReadHost){ RETURN $ReadHost }
  }
}

## ===== TITLE ===== ##
[String]$Title = "Get-NaLUNsSuperCSV"
Wr;Wr "+++++ $Title +++++" MAGENTA;Wr;Wr

## ===== CHECK THE PSTK ===== ##
If(!(Get-Module DataONTAP)){ [Void](Import-Module DataONTAP -ErrorAction SilentlyContinue) }
If(!(Get-Module DataONTAP)){ Wr "Failed to load DataONTAP PSTK!" RED;Wr;Wr;EXIT }
else{ Wr "Loaded DataONTAP PSTK." GREEN;Wr;Wr }

## ===== PROMPT FOR CONTROLLER/GET CREDENTIALS/ TEST CONNECTION ===== ##
[String]$NaController = Get-Prompt "Enter NetApp 7-Mode Controller IP/DNS:"
If(!(Get-NaCredential $NaController)){
  [String]$UserName = Get-Prompt "Enter Username:"
  Wr "Enter Password:" CYAN; $Password = Read-Host -AsSecureString
  $Credentials = New-Object System.Management.Automation.PsCredential($Username,$Password)
  [Void](Add-NcCredential -Name $NaController -Credential $Credentials)
}
If(!(Get-NaCredential $NaController)){ Wr "Failed to Get-NaCredential $NaController!" RED;Wr;Wr;EXIT }
else{ Wr ("Using credential " + ((Get-NaCredential $NaController).Credential.UserName) + " for $NaController.") GREEN;Wr }
If(!(Connect-NaController $NaController -ErrorAction SilentlyContinue)){ Wr "Failed to connect to $NaController! Verify> options httpd.admin.enable on" RED;Wr;Wr;EXIT }
else{ Wr "Connected to $NaController." GREEN;Wr;Wr }

## ===== GET LUN PROPERTIES ===== ##
Wr "Getting LUN Properties " CYAN
[System.Array]$NaLunProperties = "Path","Size","SizeUsed","MultiprotocolType","Online","Mapped","Thin","Comment","Alignment","BackingSnapshot","BlockSize","CloneBackingSnapshot","DeviceId","IsSpaceReservationEnabled","PrefixSize","ReadOnly","SerialNumber","ShareState","Staging","SuffixSize","Uuid"
[System.Array]$NaLunGeometryProperties = "BytesPerSector","Cylinders","MaxResizeSize","SectorsPerTrack","TracksPerCylinder"
[System.Array]$NaLunMapProperties = "InitiatorGroupAluaEnabled","InitiatorGroupName","InitiatorGroupOsType","InitiatorGroupPortsetName","InitiatorGroupThrottleBorrow","InitiatorGroupThrottleReserve","InitiatorGroupType","InitiatorGroupUsePartner","InitiatorGroupUuid","InitiatorGroupVsaEnabled","Initiators","LunId"

[System.Array]$LunsCollection = @()
Get-NaLun | Foreach{
  Wr "." CYAN
  $Object = New-Object PSObject
  Foreach($Property in $NaLunProperties){      
    Add-Member -InputObject $Object -MemberType NoteProperty -Name $Property -Value $_.$Property
  }
  $LunGeometry = $_ | Get-NaLunGeometry
  Foreach($Property in $NaLunGeometryProperties){
    Add-Member -InputObject $Object -MemberType NoteProperty -Name ("Geometry" + $Property) -Value $LunGeometry.$Property
  }
  [Double]$MaxGrowth = $LunGeometry.MaxResizeSize / $LunGeometry.Size
  Add-Member -InputObject $Object -MemberType NoteProperty -Name ("GeometryMaxGrowthMultiplier") -Value $MaxGrowth
  $LunMap = $_ | Get-NaLunMap
  Foreach($Property in $NaLunMapProperties){
    Add-Member -InputObject $Object -MemberType NoteProperty -Name $Property -Value $LunMap.$Property
  }
  $LunsCollection += $Object
};Wr;Wr
Wr "Saving LUN properties to $Title.CSV" CYAN;Wr;Wr
$LunsCollection | Export-CSV "$Title.CSV" -NoType


Tuesday, 22 March 2016

Managing Multiple cDOT Clusters with PowerShell - Version 2

Just over a week since releasing Version 1, I decided to rewrite the core functions, obey PowerShell’s verb naming conventions, and re-release with more functions as Version 2!

Save the script as McCmdlets.psm1 into C:\Users\{USERNAME}\Documents\WindowsPowerShell\Modules\McCmdlets and import into PowerShell with>


Import-Module McCmdlets.psm1


Note: Where cDOT cmdlets have Nc in the name, since these cmdlets are designed for managing multiple clusters, hence Mc, and with a plural in the names (not the singular.)

There is built in help>


Get-McHelp


The McCmdlets So Far:

Get-McVservers
Get-McVserversAdmin
- Get all Admin SVMs for all clusters...
Get-McVserversRunningData
- Get all Running Data SVMs for all clusters...
Get-McVolsTypeAndState
- Get Type and State of all volumes for all clusters...
Get-McVolsOffline
- Get all offline volumes for all clusters...
Get-McVolsRwOnline
- Get all R/W online volumes for all clusters...
Remove-McVolsOffline
- Remove all offline volumes in all Running Data SVMs...
Set-McUsersPasswordForAdminSVM
- Set a user’s password across all Admin SVMs...
Remove-Mc7modeSnapshots
- Remove 7-Mode Snapshots from all R/W online volumes in all Running Data SVMs...
Get-McLunsMappedStatus
- Get LUNs mapped status for all Running Data SVMs...
Get-McVolsWithLuns
- Get all volumes with LUNs for all Running Data SVMs...
Get-McVolsWithMappedLuns
- Get all volumes with Mapped LUNs for all Running Data SVMs...
Get-McVolsWithOnlyUnmappedLuns
- Get all volumes with only UnMapped LUNs for all Running Data SVMs...
Set-McVolsWithOnlyUnmappedLunsOffline
- Set all volumes with only UnMapped LUNs offline for all Running Data SVMs...

The Code

############################
## PART 1: McCmdlets Core ##
############################

Function Get-McHelp{
  Wr;Wr "+++++ Multi-cluster Cmdlets (McCmdlets) +++++" MAGENTA; Wr
  Wr;Wr "A selection of cmdlets to manage multiple clusters in one go!" CYAN; Wr
  Wr;Wr "1 " CYAN; Wr 'Import-McClusters (creates Global variable $McClusterNames)' GREEN
  If($Global:McClusterNames){ $Color = "GREEN" }
  else{ Wr;Wr;Wr "  ===== Run Import-McClusters ==== " RED; Wr; $Color = "RED" }
  Wr;Wr "2 " CYAN; Wr "Add-McCredentials" $Color
  Wr;Wr "- " CYAN; Wr "Remove-McCredentials" $Color
  If(!$Global:McClusterNames){}
  elseif(Get-McCredentials){ $Color = "GREEN" }
  else{ Wr;Wr;Wr "  ===== Run Add-McCredentials ==== " RED; Wr; $Color = "RED" }
  Wr;Wr "3 " CYAN; Wr 'Connect-McControllers (creates Global variable $McConnectedClusters)' $Color
  Wr;Wr "- " CYAN; Wr "Clear-McControllers" $Color; Wr
  Wr;Wr 'Run 1 -> 2 -> 3, then we are ready to use the below McCmdlets against $McConnectedClusters!' CYAN; Wr
  If($Global:McConnectedClusters){ $Color = "GREEN" }
  elseif($Global:CurrentNcController){ Wr 'No $McConnectedClusters, run Connect-McControllers, otherwise the below will run against the $CurrentNcController(s)!' YELLOW; Wr; $Color = "YELLOW" }
  else{ Wr 'No $McConnectedClusters and no $CurrentNcController!' RED; Wr; $Color = "RED" }
  Wr;Wr "  Get-McVservers" $Color
  Wr;Wr "  Get-McVserversAdmin" $Color
  Wr;Wr "  Get-McVserversRunningData" $Color
  Wr;Wr "  Get-McVolsTypeAndState" $Color
  Wr;Wr "  Get-McVolsOffline" $Color
  Wr;Wr "  Get-McVolsRwOnline" $Color
  Wr;Wr "  Remove-McVolsOffline" $Color
  Wr;Wr "  Set-McUsersPasswordForAdminSVM" $Color
  Wr;Wr "  Remove-Mc7modeSnapshots" $Color
  Wr;Wr "  Get-McLunsMappedStatus" $Color
  Wr;Wr "  Get-McVolsWithLuns" $Color
  Wr;Wr "  Get-McVolsWithMappedLuns" $Color
  Wr;Wr "  Get-McVolsWithOnlyUnmappedLuns" $Color
  Wr;Wr "  Set-McVolsWithOnlyUnmappedLunsOffline" $Color
  Wr;Wr      
}
Function Get-McCredentials{
  [Void](Get-DataOntapPSTK)
  If(!$Global:McClusterNames){ RETURN $FALSE }
  Foreach($Cluster in $Global:McClusterNames){ If(!(Get-NcCredential -Name $Cluster)){ RETURN $FALSE } }
  RETURN $TRUE
}
Function Get-DataOntapPSTK{
  If(!(Get-Module DataONTAP)){ [Void](Import-Module DataONTAP -ErrorAction SilentlyContinue) }
  If(!(Get-Module DataONTAP)){ RETURN $FALSE } else {  RETURN $TRUE }
}
Function Import-McClusters {
  Param([String]$File)
  If(Get-DataOntapPSTK){ Wr; Wr "PSTK loaded ..." GREEN; Wr }
  else{ Wr; Wr "Unable to load PSTK!" RED; Wr; Wr; RETURN }
  Wr; Wr "Loading file containing cluster management IPs/Name ..." CYAN; Wr; Wr
  [String]$File = Get-Prompt -Prompt "Enter Filename/Filepath" -PreAnswer $File
  If(Test-Path $File){ [System.Array]$FileContent = Get-Content $File }
  Else{ Wr; Wr "Failed to load $File!" RED; Wr; Wr; RETURN }
  If($FileContent.count -eq 0){ Wr; Wr "No content in $File!" RED; Wr; Wr; RETURN }
  [System.Array]$Global:McClusterNames = @()
  $FileContent = $FileContent | Where { ($_.Trim("`t"," ") -ne "") -and !$_.StartsWith("#") }
  $FileContent | Foreach { $Global:McClusterNames += $_.Split("#")[0].Trim("`t"," ") }
  If($Global:McClusterNames.count -eq 0){ Wr; Wr "No valid content in $File!" RED; Wr; Wr; RETURN }
  Wr; Wr "Loaded $File ..." GREEN; Wr
  Wr; Wr 'PS> $McClusterNames' CYAN; Wr; Wr
  $Global:McClusterNames; Wr
}
Function Add-McCredentials{
  Param([Switch]$Remove)
  [System.Array]$Global:McConnectedClusters = @()
  If(!$Global:McClusterNames){ Wr; Wr 'No $McClusterNames - run Import-McClusters!' RED; Wr; Wr; RETURN }
  If(!$Remove){ Wr; Wr 'Note: If there are existing stored NcCredentials for $McClusterNames, they will be overwritten (can use Get-NcCredential to check)!' YELLOW; Wr; Wr }
  If(!$Remove){ [String]$UserName = Get-Prompt -Prompt "Enter Username" }
  If(!$Remove){ Wr "Enter Password:" CYAN; $Password = Read-Host -AsSecureString }
  If(!$Remove){ $Credentials = New-Object System.Management.Automation.PsCredential($Username,$Password) }
  If(!$Remove){ $Global:McClusterNames | Foreach{ [Void](Add-NcCredential -Controller $_ -Credential $Credentials) } }
  else{ $Global:McClusterNames | Foreach{ [Void](Remove-NcCredential -Name $_ -ErrorAction SilentlyContinue) } }
  Wr; Wr 'PS> (Get-NcCredential).Name | Where{ $Global:McClusterNames -Contains $_ }' CYAN; Wr; Wr
  (Get-NcCredential).Name | Where{ $Global:McClusterNames -Contains $_ }; Wr
}
Function Remove-McCredentials{ [Void](Add-McCredentials -Remove) }
Function Connect-McControllers{
  If(!(Get-McCredentials)){ Wr; Wr 'Unable to get credentials for Clusters, have your run Import-McClusters and Add-McCredentials?' RED; Wr; Wr; RETURN }
  [System.Array]$Global:McConnectedClusters = @(); Wr
  $Global:McClusterNames | Foreach {
    $Connected = Connect-NcController -Name $_ -Timeout 15000 -ErrorAction SilentlyContinue
    If($Connected){ $Global:McConnectedClusters += $_; Wr "Connected to $_!" GREEN; Wr }       
    Else{ Wr "Unable to connect to $_!" RED; Wr }
  }
  $Global:CurrentNcController = $null
  $Global:McConnectedClusters | Foreach { [Void](Connect-NcController -Name $_ -Add -Timeout 15000 -ErrorAction SilentlyContinue) }
  Wr; Wr 'Connected to the $McConnectedClusters below:' GREEN; Wr
  Wr 'PS> $CurrentNcController | FT Name,Address,Vserver,Version -AutoSize' CYAN; Wr
  $Global:CurrentNcController | FT Name,Address,Vserver,Version -AutoSize
}
Function Clear-McControllers{
  [System.Array]$Global:McConnectedClusters = @()
  $Global:CurrentNcController = $null
}
Function Wr{
  Param([String]$Echo,[String]$Ink = "WHITE")
  If($Echo){ Write-Host $Echo -ForegroundColor $Ink -NoNewLine }else{ Write-Host }
}
Function Get-Prompt{
  Param([String]$Prompt,[System.Array]$Answers,[String]$PreAnswer,[Switch]$CaseSensitive,[Switch]$AllowCR)
  While ($true){
    Wr ($Prompt + ":") CYAN
    If($PreAnswer){ Wr $PreAnswer; Wr; RETURN $PreAnswer }else{ $ReadIn = Read-Host }
    If($Answers -and  $CaseSensitive){ Foreach ($a in $Answers){ If($ReadIn -ceq $a){ RETURN $ReadIn } } }
    If($Answers -and !$CaseSensitive){ Foreach ($a in $Answers){ If($ReadIn -eq  $a){ RETURN $ReadIn } } }
    If(!$Answers -and ($ReadIn.length -gt 0) ){ RETURN $ReadIn }
    If($AllowCR  -and ($ReadIn.length -eq 0) ){ RETURN "" }
  }
}

###############################
## PART 2: McCmdlets Cmdlets ##
###############################

## ===== McVservers SECTION ===== ##
Function Get-McVservers {
  Param([System.Object]$NcController,[Switch]$Admin,[Switch]$RunningData,[Switch]$NoFormat)
  If(!$Global:CurrentNcController){ RETURN }
  # ===== SVM ATTRIBUTES ===== #
  $VsAttrs = Get-NcVserver -Template
  $VsAttrs.Comment          = ""
  $VsAttrs.OperationalState = ""
  $VsAttrs.VserverType      = ""
  # ===== ACQUIRE DATA ===== #
  If(!$NcController){ [System.Object]$SVMs = Get-NcVserver -Attributes $VsAttrs }
  else{               [System.Object]$SVMs = Get-NcVserver -Controller $NcController -Attributes $VsAttrs }
  # ===== FUNCTION OUTPUT ===== #
  If($Admin){           $SVMs = $SVMs | Where{ $_.VserverType -eq "admin" } }
  elseif($RunningData){ $SVMs = $SVMs | Where{ ($_.OperationalState -eq "running") -and ($_.VserverType -eq "data") } }
  [System.Array]$TableFormat = @()
  $TableFormat += @{label='Cluster';e={$_.NcController.Name}}
  $TableFormat += @{label='Vserver';e={$_.VserverName}}
  $TableFormat += @{label='State';e={$_.OperationalState}}
  $TableFormat += @{label='Type';e={$_.VserverType}}
  $TableFormat += "Comment"
  If($NoFormat){ $SVMs }
  else{ $SVMs | FT $TableFormat -AutoSize }
}
Function Get-McVserversAdmin{ Get-McVservers -Admin }
Function Get-McVserversRunningData{ Get-McVservers -RunningData }

## ====== McVols SECTION ===== ##
Function Get-McVolsTypeAndState {
  Param([System.Object]$NcController,[String]$Vserver,[Switch]$Offline,[Switch]$RwOnline,[Switch]$NoFormat)
  If(!$Global:CurrentNcController){ RETURN }
  # ===== VOLUME ATTRIBUTES ===== #
  $VolAttrs = Get-NcVol -Template
  Initialize-NcObjectProperty -Object $VolAttrs -Name VolumeIdAttributes
  Initialize-NcObjectProperty -Object $VolAttrs -Name VolumeStateAttributes
  $VolAttrs.VolumeIdAttributes.Comment  = ""
  $VolAttrs.VolumeIdAttributes.Type     = ""
  $VolAttrs.VolumeStateAttributes.State = ""
  # ===== ACQUIRE DATA ===== #
  If(!$NcController){ [System.Object]$Vols = Get-NcVol -Attributes $VolAttrs }
  elseif(!$Vserver){  [System.Object]$Vols = Get-NcVol -Controller $NcController -Attributes $VolAttrs }
  else{               [System.Object]$Vols = Get-NcVol -Controller $NcController -VserverContext $Vserver -Attributes $VolAttrs  }
  # ===== FUNCTION OUTPUT ===== #
  If($Offline){      $Vols = $Vols | Where{ $_.VolumeStateAttributes.State -eq "offline" } }
  elseif($RwOnline){ $Vols = $Vols | Where{ ($_.VolumeIdAttributes.Type -eq "rw") -and ($_.VolumeStateAttributes.State -eq "online") } }
  [System.Array]$TableFormat = @()
  $TableFormat +=    @{label='Cluster';e={$_.NcController.Name}}
  $TableFormat += "Vserver"
  $TableFormat += "Name"
  $TableFormat += @{label='Type';e={$_.VolumeIdAttributes.Type}}
  $TableFormat += @{label='State';e={$_.VolumeStateAttributes.State}}
  $TableFormat += @{label='Comment';e={$_.VolumeIdAttributes.Comment}}
  If($NoFormat){ $Vols }
  else{ $Vols | FT $TableFormat -AutoSize }
}
Function Get-McVolsOffline{  Get-McVolsTypeAndState -Offline }
Function Get-McVolsRwOnline{ Get-McVolsTypeAndState -RwOnline }
Function Remove-McVolsOffline{
  Param([String]$OverrideFile,[Switch]$NoPrompt)
  # ===== HANDLE THE OVERRIDE FILE ===== #
  [System.Array]$Overrides = @() 
  If($OverrideFile -and (Test-Path $OverrideFile)){
    [System.Array]$FileContent = Get-Content $OverrideFile
    $FileContent = $FileContent | Where { ($_.Trim("`t"," ") -ne "") -and !$_.StartsWith("#") }
    $FileContent | Foreach { $Overrides += $_.Split("#")[0].Trim("`t"," ") }
  }elseif($OverrideFile){ Wr;Wr "Failed to load specified override file $OverrideFile!" RED;Wr;Wr;RETURN }
  # ===== REMOVE OFFLINES VOLUMES NOT IN OVERRIDES ===== #
  $Global:CurrentNcController | Foreach {
    Foreach($SVM in (Get-McVservers $_ -RunningData -NoFormat) ){
      Foreach($Volume in (Get-McVolsTypeAndState $_ $SVM.VserverName -Offline -NoFormat) ){
        If($Overrides -NotContains $Volume.Name){
          If(!$NoPrompt){ $Volume | Remove-NcVol }
          Else{ $Volume | Remove-NcVol -Confirm:$FALSE }
        }
      }
    }
  }
}

## ===== McUsers SECTION ===== ##
Function Set-McUsersPasswordForAdminSVM{
  If(!$Global:CurrentNcController){ RETURN }; Wr
  [String]$UserName = Get-Prompt -Prompt "Enter user whose password is to be reset"
  Wr "Enter new password:" CYAN; $Password = Read-Host -AsSecureString
  $NewCred = New-Object System.Management.Automation.PsCredential($UserName,$Password); Wr
  $Global:CurrentNcController | Foreach {
    $Vserver = (Get-McVservers $_ -Admin -NoFormat).VserverName
    [System.Array]$GetUser = Get-NcUser -Name $UserName -Vserver $Vserver -Controller $_ -ErrorAction SilentlyContinue
    If(!$GetUser){ Wr ("User $UserName does not exist on " + $_.Name + ":" + $Vserver + "!") RED; Wr }
    elseif($GetUser[0].AuthMethod -ne "password"){ Wr ("User $UserName is not password authenticated on " + $_.Name + ":" + $Vserver + "!") RED; Wr }
    else{
      [Void](Set-NcUserPassword -Credential $NewCred -VserverContext $Vserver -Controller $_)
      Wr ("Ran Set-NcUserPassword for user $UserName on " + $_.Name + ":" + $Vserver + "!") GREEN; Wr
    }
  }; Wr
}

## ===== Mc7modeSnapshots SECTION ===== ##
Function Remove-Mc7modeSnapshots{
  If( !$Global:CurrentNcController){ RETURN }; Wr
  $SnapAttrs = Get-NcSnapshot -Template
  $SnapAttrs.Is7ModeSnapshot = ""
  $Global:CurrentNcController | Foreach {
    Foreach($SVM in (Get-McVservers $_ -RunningData -NoFormat) ){
      Foreach($Volume in (Get-McVolsTypeAndState $_ $SVM.VserverName -RwOnline -NoFormat) ){   
        Wr ("Removing any 7-Mode snapshots from " + $_.Name + ":" + $SVM.VserverName + "\" + $Volume.Name) YELLOW; Wr
        $SevenModeSnapshots = Get-NcSnapshot -Volume $Volume.Name -Attributes $SnapAttrs -Controller $_ -Vserver $SVM.VserverName | Where { $_.Is7ModeSnapshot -eq $TRUE }
        $SevenModeSnapshots | Format-Table Name,Is7ModeSnapshot -AutoSize
        $SevenModeSnapshots | Where { $_.Is7ModeSnapshot -eq $TRUE } | Remove-NcSnapshot -Confirm:$FALSE
      }
    } 
  };Wr
}

## ===== McLuns SECTION ===== ##
Function Get-McLunsMappedStatus{
  Param([Switch]$NoFormat)
  If(!$Global:CurrentNcController){ RETURN }
  $LunAttrs = Get-NcLun -Template
  $LunAttrs.Mapped = ""
  [System.array]$LunArray = @()
  $Global:CurrentNcController | Foreach {
    Foreach($SVM in (Get-McVservers $_ -RunningData -NoFormat) ){
      Foreach($Volume in (Get-McVolsTypeAndState $_ $SVM.VserverName -RwOnline -NoFormat) ){   
        $LUNs = Get-NcLun -Volume $Volume.Name -Attributes $Attrs -VserverContext $SVM.VserverName -Controller $_
        Foreach($LUN in $LUNs){ $LunArray += $LUN }
      }
    }
  }
  If($NoFormat){ $LunArray }
  else{ $LunArray | FT @{label='Cluster';e={$_.NcController.Name}},Vserver,Mapped,Volume,Path -AutoSize }
}
Function Get-McVolsWithLuns{
  Param([Switch]$VolsWithMappedLuns)
  [System.Array]$ObjectCollection = @()
  If($VolsWithMappedLuns){ $GetMcLuns = Get-McLunsMappedStatus -NoFormat | Where { $_.Mapped -eq $TRUE } }
  else{ $GetMcLuns = Get-McLunsMappedStatus -NoFormat }
  $GetMcLuns | Foreach{
    $Object = New-Object PSObject
    Add-Member -InputObject $Object -MemberType NoteProperty -Name Cluster -Value ($_.NcController.Name)
    Add-Member -InputObject $Object -MemberType NoteProperty -Name Vserver -Value ($_.Vserver)
    Add-Member -InputObject $Object -MemberType NoteProperty -Name Volume  -Value ($_.Volume)
    If( !($ObjectCollection -Match $Object) ){ $ObjectCollection += $Object }
  }
  $ObjectCollection
}
Function Get-McVolsWithMappedLuns{ Get-McVolsWithLuns -VolsWithMappedLuns }
Function Get-McVolsWithOnlyUnmappedLuns{
  $VolsWithMappedLuns = Get-McVolsWithMappedLuns
  [System.Array]$ObjectCollection = @()
  Get-McVolsWithLuns | Foreach{
    If( !($VolsWithMappedLuns -Match $_) ){ $ObjectCollection += $_ }
  }
  $ObjectCollection
}
Function Set-McVolsWithOnlyUnmappedLunsOffline{
  Foreach($Vol in Get-McVolsWithOnlyUnmappedLuns){
    $Controller = $Global:CurrentNcController | Where{ $_.Name -eq $Vol.Cluster }
    Set-NcVol -Offline -Name $Vol.Volume -VserverContext $Vol.Vserver -Controller $Controller
  }
}