Wednesday, 8 February 2017

Get-NcCommands ONTAP Data Collector: Part 1 of 2 (Collect-NcData.ps1)

It’s very useful to collect the output of Get-NcCOMMANDs, and save as XML so you’ve got the wealth of data available via the Data ONTAP PS Toolkit at your fingertips, so you can process the XMLs (via Import-CLIXML) any way and any time you like, and don’t need to keep firing API calls at the Cluster to get information. The XMLs could even be used for configuration backup/recall purposes.

Since there are so many Get-NcCOMMANDs (486 in the 4.1 toolkit), I wondered how easy it is to just try and grab everything. It’s not quite as easy as just firing every Get-NcCOMMAND at the cluster, since some need more input, some need to be directed at the Vserver, and other reasons (the program only handles the easy ones, and informs on the others.)

The following PowerShell script collects all the Get-NcCOMMANDs possible in minimal lines of PowerShell. I added options like Manual Exclusions (ManualExclude.CSV) for things you’re not interested in, or commands that you know are slow, or commands that need to be broken up further (like Get-NcVol on large clusters). After a first run it will generate a list of Automatic Exclusions (AutoExclude.CSV), this is things that need more input (on the first run, press enter where prompted, the program will exclude these from the next run) and things that error. And there are various other outputs as detailed in the comments in the script.

As always you’re going to want to test this against lab/DR boxes first. And for complete data collection, we would need to run additional Get-NcCOMMANDs (especially for those commands which require Vserver context - this will be in part 2).

Image: Collect-NcData.ps1 and Files

The Script


## INPUTS (MANDATORY): ##
Param(
  [Parameter(Mandatory=$True)][String]$Cluster,
  [Parameter(Mandatory=$True)][String]$UserName,
  [Parameter(Mandatory=$True)][Security.SecureString]$Password
  # TIP: Run in PS> $P = Read-Host -AsSecureString
  # You can use this after -Password
)
## INPUTS (OPTIONAL): ##
# ~ManualExclude.CSV: Headers of ManualExclude = NAME,...
# This can be in the runtime folder \ for global application ...
# ... or \$Cluster folder for $Cluster application

## OUTPUTS: ##
# ~All.Get-NCs.CSV - A complete list of all Get-NCs
# $Cluster\~AutoExclude.CSV - Get-NCs detected as needing more input or erroring
# $Cluster\~Timings.CSV - used for timing how long the Get-Ncs take
# $Cluster\~Processed.CSV - the Get-NCs that were processed
# $Cluster\~ScreenOutput.TXT - screen output recording
# $Cluster\$Cluster.$GetNc.XML - for each Get-Nc
# N.B. AutoExclude.CSV is created on first run. Press enter where prompted for more input.

## FUNCTION: Collect-NcData ##
Function Collect-NcData{
  Param([Parameter(Mandatory=$True)][String]$Cluster)
 
  ## GET ALL GET-NCs and CREATE CSV ##
  $GetNcINFO = Get-NcHelp | where {$_.Name.StartsWith("Get-Nc")}
  [System.Array]$AllGetNCs = "NAME,CATEGORY"
  $GetNcINFO | Foreach{ $AllGetNCs += ($_.Name + "," + $_.Category) }
  $AllGetNCs | Out-File "~All.Get-NCs.CSV" -Encoding Default -Force
 
  ## CAPTURE SCREEN OUTPUT TO $Cluster\~ScreenOutput.txt ##
  $ErrorActionPreference = "SilentlyContinue"
  Stop-Transcript | out-null
  $ErrorActionPreference = "Continue"
  [Void](New-Item "$Cluster" -ItemType Directory -Force)
  If(!(Test-Path $Cluster)){ ("Unable to access path " + $pwd.path + " \$Cluster"); RETURN }
  Start-Transcript -path "$Cluster\~ScreenOutput.TXT"
  # If you terminate early, remember PS> stop-transcript!
 
  ## ACQUIRE EXCLUSIONS and GET-NCs to GATHER ##
  [System.Array]$Exclude = @()
  "~ManualExclude.CSV","$Cluster\~ManualExclude.CSV","$Cluster\~AutoExclude.CSV" | Foreach {
    If(Test-Path "$_"){ $Exclude += (Import-CSV "$_")."NAME" }
  }
  [System.Array]$GetNCs = ($GetNcINFO).Name
  If($Exclude){ $GetNCs = (Compare-Object $GetNCs $Exclude).InputObject }
  [System.Array]$AutomaticExclusions = @()
  If(Test-Path "$Cluster\~AutoExclude.CSV"){[System.Array]$AutomaticExclusions = Import-CSV "$Cluster\~AutoExclude.CSV"}
  [System.Array]$AutoExclude = "NAME,REASON"
  If($AutomaticExclusions){ $AutomaticExclusions | Foreach { $AutoExclude += ($_."NAME" + "," + $_."REASON") } }
 
  ## COLLECT DATA ##
  [System.Array]$TimingsOutput = "NAME,Time(Secs)"
  Foreach($GetNc in $GetNCs){
    $Time1 = Get-Date -uformat %s
    "INFO_: Collecting $GetNc"
    [System.Array]$NcData = Invoke-Expression $GetNc
    If($NcData){ $NcData | Export-CLIXML "$Cluster\$Cluster.$GetNc.XML" }
    $TimingsOutput += "$GetNc," + [math]::Round(((Get-Date -uformat %s) - $Time1),2)
  }
  "INFO_: Finished data collection"
  $TimingsOutput | Out-File "$Cluster\~Timings.CSV" -Encoding Default -Force
  Stop-Transcript
 
  ## ANALYZE SCREEN OUTPUT (ASO) ##
  # This works on the simple principle that successful data collection sees an INFO_ following an INFO_.
  [System.Array]$Processed = "NAME"
  [System.Array]$ASO = Get-Content "$Cluster\~ScreenOutput.TXT"
  For($i = 0; $i -lt ($ASO.count -1); $i++){
    If($ASO[$i].StartsWith("INFO_: Collecting")){
      [String]$GetNc = $ASO[$i].Split(" ")[2]
      If($ASO[$i+1].StartsWith("INFO_: Collecting")){ $Processed += $GetNc }
      elseif($ASO[$i+1] -eq "INFO_: Finished data collection"){ $Processed += $GetNc }
      else{
        [String]$Reason = "Auto Exclude"
        If($ASO[$i+1].Contains("Unable to find API")){ $Reason = "Unable to find API" }
        If($ASO[$i+3].Contains("must be directed to a data vserver")){ $Reason = "Direct to Data Vserver" }
        $AutoExclude += "$GetNc,$Reason"
      }
    }
  }
  $AutoExclude | Out-File "$Cluster\~AutoExclude.CSV" -Encoding Default -Force
  $Processed | Out-File "$Cluster\~Processed.CSV" -Encoding Default -Force
}

## GENERIC: Load PSTK ##
"INFO_: Loading the Data ONTAP PSTK"
If(!(Get-Module DataONTAP)){ [Void](Import-Module DataONTAP -ErrorAction SilentlyContinue) }
If(!(Get-Module DataONTAP)){ "ERROR: Failed to load Data ONTAP PSTK"; EXIT }

## GENERIC: Connect to Cluster ##
"INFO_: Connecting to $Cluster"
$Cred = New-Object System.Management.Automation.PsCredential($UserName,$Password)
If(!(Test-Connection $Cluster)){ "ERROR: Cannot ping to $Cluster"; EXIT }
$Global:CurrentNcController = $NULL
[Void](Connect-NcController -Name $Cluster -Credential $Cred)
If(!$Global:CurrentNcController){ "ERROR: Cannot connect to $Cluster"; EXIT }

## MAIN PROGRAM ##
Collect-NcData $Cluster


No comments:

Post a Comment