Thursday, 15 February 2018

SolidFire Resources

The links below come mostly from the NetApp SolidFire Resources page (which is here and requires you to be first logged into mysupport.netapp.com, but only one of the links below actually requires login.) I’ve added a few more links and formatted it my way. If you’re wondering why I’d post this page (it does seem a tad like duplication), it’s because, when I start learning something new to me, I like to be aware of lots (all) of the collateral, so I can pick and choose what I want to start reading/watching first (usually via an iPad from the comfort of my favourite armchair) and then consume as much knowledge as I need.

Image: NetApp love SolidFire

Introduction

PDFs from www.netapp.com/us/media:

Videos from YouTube:


Installing and Deploying NetApp SolidFire

PDFs from www.netapp.com/us/media:

Videos from tv.netapp.com:

PDFs from library.netapp.com (requires sign-in to mysupport.netapp.com):


Integrating NetApp SolidFire into your Environment
+ all the SolidFire TRs

PDFs from www.netapp.com/us/media:

Tuesday, 13 February 2018

How Many Items does Read-NcDirectory Return?

I had an email conversation with a colleague regards something I had written. Specifically - http://www.cosonok.com/2015/07/search-ncfilesystemps1-cdot.html - which morphed a year-and-a-half later into the better - http://www.cosonok.com/2017/02/treesize-for-clustered-ontap-get.html. My colleague commented “you use the read-directory, but that doesn’t return all records (only first 2000 or so)” so I had to prove this was not the case.

In the following post - all using PowerShell and the Data ONTAP PowerShell Toolkit - I set up a test SVM, test volume, and create 100’000 files in my test volume, and then run Read-NcDirectory on that volume to see how many results come back.


PS> Import-Module DataONTAP
PS> Connect-NcController 10.9.3.10
PS> Get-NcVserver
PS> Get-NcAggr

PS> New-NcVserver -Name SVM_TEST -RootVolume ROOTVOL -RootVolumeAggregate data1 -RootVolumeSecurityStyle UNIX

PS> New-NcVol TEST_VOL data1 10g -JunctionPath /TEST_VOL -VserverContext SVM_TEST


The next bit will create the 100’000 files in the volume.


PS> For($i=1;$i -le 100000;$i++){Write-Host "$i " -NoNewLine; Write-NcFile "/vol/TEST_VOL/file$i" -Data "$i" -VserverContext SVM_TEST}


It will take some time to create 100’000 files (a couple of hours on my VSIM). You can check the filecount is increasing by running df -i on the Clustershell CLI.

And when that’s finished we see if Read-NcDirectory can return the 100’000+ records (Read-NcDirectory will take a while to process).


PS> $R = Read-NcDirectory -path /vol/TEST_VOL -VserverContext SVM_TEST
PS> $R.count
100003


And visual proof!

Image: Read-NcDirectory returning > 100’000 records - Picture 1

Image: Read-NcDirectory returning > 100’000 records - Picture 2: $R Output

I could keep running my test with ever larger file count and see if there is a limit, but for now we’ll leave it at “Read-NcDirectory definitely returns over 100’000 records, and I don’t know if there is a limit.”

Monday, 12 February 2018

NetApp Technical Report (TR) PowerShell Collector Script: Version 2.1

A minor update of the NetApp Technical Report (TR) PowerShell Collector Script: Version 2. I’ve been doing a bit of blog house keeping and moved the NetApp TR catalogue into dedicated pages, and have the links to these pages in the sidebar. The script is updated to get the PDF titles from the new pages. The updated script allows me to have multiple source pages, rather than one big one. The new pages have the HTML links to the TRs.

Image: Pages Sidebar (standard/non-mobile view)

The Script


##########################################
## NetAppTR_Collector.ps1 (Version 2.1) ##
##########################################

# This program will collect all the TRs from -
# https://www.netapp.com/us/media/tr-XXXX.pdf
# - where XXXX is specified by $RangeStart to $RangeEnd
# Version 2 adds:
# 1) Checks for file existence on the TR-XXXX part (not the whole file name)
# 2) Automatically names the PDFs from http://www.cosonok.com/2017/07/netapp-technical-reports-catalogue.html
# 3) Logs updated files and new files (as well as dead links - the log filename has been changed)
# Version 2.1:
# 1) PDF names now come from:
# http://www.cosonok.com/p/netapp-trs-4500-4999.html
# http://www.cosonok.com/p/netapp-trs-4000-4499.html

Param(
  [Int]$RangeStart = 4000,
  [Int]$RangeEnd   = 4700,
  [String]$DownloadFolder = "C:\Downloads\NetAppTRs\",
  [String]$LogFile = "NetAppTR_Collector.log"
)

# Checks if download folder path has \ at the end:
If($DownloadFolder -match '.+?\\$'){}
else{$DownloadFolder += "\"}

[String]$LogFilePath = $DownloadFolder + $LogFile
[System.Array]$CatalogURLs = @()
$CatalogURLs += "http://www.cosonok.com/p/netapp-trs-4500-4999.html"
$CatalogURLs += "http://www.cosonok.com/p/netapp-trs-4000-4499.html"

# Generic display function:
Function Wr{Param($P,$I="WHITE");Write-Host $P -ForegroundColor $I}

##########################
## TITLE EXTRACTER CODE ##
##########################

[System.Array]$URLdata = @()
Foreach($URL in $CatalogURLs){
  Try{$WebData = Invoke-Webrequest $URL -Method Get}
  Catch{
    Wr "FAILED: Unable to acquire catalog from $URL!" RED;PAUSE;EXIT
  }
  $URLdata += $WebData
}

## Get-Title function:
Function Get-Title{
  Foreach($WebPage in $URLdata){
    [System.Array]$Arr = $WebPage.Content.Split("`n")
    $Catalog = $WebPage.Content
    $Count = $Arr.Count
    $Position = 0
    For($k=$Position;$k -lt $Count;$k++){
      If($Arr[$k].Contains("href") -and $Arr[$k].Contains("TR-$($j):")){
        $Position++
        [String]$Title = $Arr[$k].Split(">")[1]
        $l=1
        While($TRUE){
          If($Title.Contains("<")){
            $Title = $Title.Split("<")[0]
            RETURN $Title
          }else{
            $Title += " "
            $Title += $Arr[$k+$l]
          }
          $l++
          If(($k+$l) -ge $Count){
            Wr "FAILED: Something went wrong here!" RED;PAUSE;EXIT
          }
        }
      }
    }
  }
  RETURN "TR-$($j)"
}

## Create hashtable of titles to number XXXX:
[System.Object]$Titles = @{}
For($j=$RangeEnd;$j -ge $RangeStart;$j--){
  [String]$Titles."$j" = "TR-$($j)"
  $Titles."$j" = (Get-Title).Replace(":"," -").Replace("/",",")
}

#################################
## PREPARE DOWNLOAD FOLDER/LOG ##
#################################

[Void](New-Item -ItemType Directory -Force -Path $DownloadFolder)
If(Test-Path $DownloadFolder){}
else{Wr "FAILED: Test-Path $DownloadFolder" RED;PAUSE;EXIT}
If(Test-Path $LogFilePath){
  Copy-Item $LogFilePath ($LogFilePath + ".bak")
}else{
  [Void](New-Item $LogFilePath -type file -force)
}

######################
## PDF RENAMER CODE ##
######################

Get-ChildItem $DownloadFolder | Foreach{
  $CIname = $_.Name
  If($CIname.StartsWith("TR-","CurrentCultureIgnoreCase")){
    [String]$TRnumber = $CIname.Split("-")[1]
    $TRnumber = $TRnumber.Substring(0,4)   
    If($Titles.$TRnumber){
      If($CIname -ne ($Titles.$TRnumber + ".pdf")){
        Wr "Renaming item: $($DownloadFolder)$($CIname) to $($Titles.$TRnumber).pdf"
        Rename-Item ($DownloadFolder + $CIname) ($Titles.$TRnumber + ".pdf")
      }
    }
  }
}

#######################
## TR COLLECTOR CODE ##
#######################

## Download the TRs:
Import-Module BitsTransfer
For($i=$RangeStart; $i -le $RangeEnd; $i++){
  $Title = "tr-$i"
  $WebUrl = "https://www.netapp.com/us/media/$Title.pdf"
  $SavePath = ($DownloadFolder + $Titles."$i" + ".pdf")

  ## Get the header:
  $hdr = $NULL
  Try{$hdr = Invoke-WebRequest $WebUrl -Method Head}
  Catch{"Dead link: $WebUrl" | Out-File $LogFilePath -Append}

  ## If we have a header:
  If($hdr){

    ## Only download updated versions:
    $Download = $FALSE     
    If(Test-Path $SavePath){
      $SavedLRT = [DateTime]((Get-ChildItem $SavePath).LastWriteTime)
      $WebFileLRT = [DateTime]($hdr.headers."last-modified")
      If($WebFileLRT.Ticks -gt $SavedLRT.Ticks){
        Wr "UPDATED   : $Title" GREEN
        "Updated  : $Title" | Out-File $LogFilePath -Append
        $Download = $TRUE
      }else{
        Wr "NO UPD.   : $Title"
      }
    }else{
      Wr "NEW D/LOAD: $Title" GREEN
      "New D/L  : $Title" | Out-File $LogFilePath -Append
      $Download = $TRUE
    }

    ## Download:
    If($Download){
      Start-BitsTransfer -Source $WebUrl -Destination $SavePath
    }
  }else{
    Wr "DEAD LINK : $WebUrl"
  }
}

PAUSE


Sunday, 11 February 2018

How to Upgrade CN1610 Firmware from 1.1.0.X to 1.2.0.7

Something I was involved with towards the end of last year, upgrading a load of production NetApp CN1610 cluster switches - with various versions of FASTPATH 1.1.0.X - to 1.2.0.7. I had to document a procedure. The upgrade procedure here is slightly different to the official procedure (which I think is a bit convoluted, and injects a risk of the wrong switch being rebooted and causing an outage). This is easy to do, non-disruptive, and shouldn’t take longer than 2 hours.

Image: CN1610 rear view

Preparation

- Run Config Advisor and verify no issues are flagged regards cluster cabing
- Run the following commands from Clustershell to double-check that each node has a connection to both cluster switches::>


node run * options cdpd.enable


If cdpd is not enabled, enable it temporarily. After enabling cdpd it may take a few minutes to discover neighbors::>


node run * options cdpd.enable on


Inspect the show-neighbors outputs::>


node run * cdpd show-neighbors
node run * cdpd show-neighbors -v


If cdpd was previously not enabled, disable it::>


node run * options cdpd.enable off


- Verify the cluster ping-cluster output::>


set advanced
cluster ping-cluster -node NODE1
set admin


You should have no basic connectivity failures and ‘Larger than PMTU communication’ success on all paths.
Note: It is sufficient to run ping-cluster on one node (takes a while to run). If issues are detected, run ping-cluster against the node(s) with issue.

- Acquire login credentials to the cluster switches

Pre-Requisites

1) TFTP Server for uploading firmware/RCF to the switch (I personally use 'Solarwinds Free TFTP Server')
2) Required software:

From the NetApp CN1610 Cluster Switch Software Download Instructions page -
- download:

- FASTPATH 1.2.0.7 for CN1610 (NetApp_CN1610_1.2.0.7.stk)
- RCF 1.2 (this is the text file CN1610_CS_RCF_v1.2.txt)

Upgrading Cluster Switch Firmware and RCF

1) Pre-Upgrade Clustershell Tasks

1.1) Send AutoSupports::>


autosupport invoke -node * -type all -message "MAINT=2h Upgrading Cluster Switch Firmware"


1.2) Check Cluster Health (all nodes should be healthy)::>


cluster show


1.3) Check Cluster Ports (all cluster ports should be up)::>


net port show -role cluster


1.4) Check Cluster LIFs (all cluster LIFs should be home)::>


net int show -role cluster


1.5) Verify that Cluster LIFs are set to auto-revert (auto-revert should be set to true)::>


net int show -role cluster -fields auto-revert


2) Upgrading Cluster Switch 1 Firmware and RCF

2.1) Connect to cluster switch 1

Note: To find the cluster switch management address use the command ‘show serviceport

2.2) To check current FASTPATH and RCF versions, run the below commands:


(CN1610-SW1) > enable
(CN1610-SW1) # show version
(CN1610-SW1) # show running-config


Note: The RCF version is listed in the description for interface 3/64 in the ‘show running-config’ output.

2.3) To upgrade FASTPATH, the commands are:


(CN1610-SW1) # show bootvar
(CN1610-SW1) # copy active backup


This will take a few minutes...


(CN1610-SW1) # show bootvar
(CN1610-SW1) # copy tftp://{YOUR_TFTP_SERVER}/NetApp_CN1610_1.2.0.7.stk active


This will take a few minutes...


(CN1610-SW1) # show bootvar


The next-active boot-image version should show as 1.2.0.7.

2.4) If the RCF version was not 1.2 (most likely it will be 1.1), you need to upgrade the RCF - the commands to do this are:


(CN1610-SW1) # show running-config config_backup.scr
(CN1610-SW1) # copy tftp://{YOUR_TFTP_SERVER}/CN1610_CS_RCF_v1.2.txt nvram:script CN1610_CS_RCF_v1.2.scr
(CN1610-SW1) # script list
(CN1610-SW1) # script apply CN1610_CS_RCF_v1.2.scr
(CN1610-SW1) # show running-config
(CN1610-SW1) # show port-channel 3/1


Note: The RCF version is listed in the description for interface 3/64 in the ‘show running-config’ output.

2.5) Save running configuration (so it becomes the startup configuration when you reboot) and reboot cluster switch 1:


(CN1610-SW1) # write memory
(CN1610-SW1) # reload


SWITCH REBOOTS

2.6) After reboot, verify FASTPATH and RCF versions have been updated, and the port-channel is working:


(CN1610-SW1) > enable
(CN1610-SW1) # show version
(CN1610-SW1) # show running-config
(CN1610-SW1) # show port-channel 3/1


Note: The RCF version is listed in the description for interface 3/64 in the ‘show running-config’ output.

2.7) Additionally: verify that SSH is enabled (RCF 1.2 disables telnet):


(CN1610-SW1) # ip ssh protocol 2
(CN1610-SW1) # config
(CN1610-SW1) (Config)# crypto key generate rsa
(CN1610-SW1) (Config)# crypto key generate dsa
(CN1610-SW1) (Config)# exit
(CN1610-SW1) # ip ssh server enable
(CN1610-SW1) # show ip ssh
(CN1610-SW1) # write memory
(CN1610-SW1) # exit
(CN1610-SW1) > logout


3) Confirm the cluster is healthy after the Cluster Switch 1 upgrade

3.1) Check Cluster Health (all nodes should be healthy)::>


cluster show


3.2) Check Cluster Ports (all cluster ports should be up)::>


net port show -role cluster


3.3) Check Cluster LIFs (all cluster LIFs should be home)::>


net int show -role cluster


4) Upgrading Cluster Switch 2 Firmware and RCF

4.1) Connect to cluster switch 2

Note: To find the cluster switch management address use the command ‘show serviceport

4.2) To check current FASTPATH and RCF versions, run the below commands:


(CN1610-SW2) > enable
(CN1610-SW2) # show version
(CN1610-SW2) # show running-config


Note: The RCF version is listed in the description for interface 3/64 in the ‘show running-config’ output.

4.3) To upgrade FASTPATH, the commands are:


(CN1610-SW2) # show bootvar
(CN1610-SW2) # copy active backup


This will take a few minutes...


(CN1610-SW2) # show bootvar
(CN1610-SW2) # copy tftp://{YOUR_TFTP_SERVER}/NetApp_CN1610_1.2.0.7.stk active


This will take a few minutes...


(CN1610-SW2) # show bootvar


The next-active boot-image version should show as 1.2.0.7.

4.4) If the RCF version was not 1.2 (most likely it will be 1.1), you need to upgrade the RCF - the commands to do this are:


(CN1610-SW2) # show running-config config_backup.scr
(CN1610-SW2) # copy tftp://{YOUR_TFTP_SERVER}/CN1610_CS_RCF_v1.2.txt nvram:script CN1610_CS_RCF_v1.2.scr
(CN1610-SW2) # script list
(CN1610-SW2) # script apply CN1610_CS_RCF_v1.2.scr
(CN1610-SW2) # show running-config
(CN1610-SW2) # show port-channel 3/1


Note: The RCF version is listed in the description for interface 3/64 in the ‘show running-config’ output.

4.5) Save running configuration (so it becomes the startup configuration when you reboot) and reboot cluster switch 2:


(CN1610-SW2) # write memory
(CN1610-SW2) # reload


SWITCH REBOOTS

4.6) After reboot, verify FASTPATH and RCF versions have been updated, and the port-channel is working:


(CN1610-SW2) > enable
(CN1610-SW2) # show version
(CN1610-SW2) # show running-config
(CN1610-SW2) # show port-channel 3/1


Note: The RCF version is listed in the description for interface 3/64 in the ‘show running-config’ output.

4.7) Additionally: verify that SSH is enabled (RCF 1.2 disables telnet):


(CN1610-SW2) # ip ssh protocol 2
(CN1610-SW2) # config
(CN1610-SW2) (Config)# crypto key generate rsa
(CN1610-SW2) (Config)# crypto key generate dsa
(CN1610-SW2) (Config)# exit
(CN1610-SW2) # ip ssh server enable
(CN1610-SW2) # show ip ssh
(CN1610-SW2) # write memory
(CN1610-SW2) # exit
(CN1610-SW2) > logout


5) Post-Upgrade ClusterShell Tasks

5.1) Send AutoSupports::>


autosupport invoke -node * -type all -message MAINT=END


5.2) Check Cluster Health (all nodes should be healthy)::>


cluster show


5.3) Check Cluster Ports (all cluster ports should be up)::>


net port show -role cluster


5.4) Check Cluster LIFs (all cluster LIFs should be home)::>


net int show -role cluster


5.5) Verify the cluster network with cluster ping-cluster::>


set advanced
cluster ping-cluster -node NODE1
set admin


You should have no basic connectivity failures and ‘Larger than PMTU communication’ success on all paths.
Note: It is sufficient to run ping-cluster on one node (takes a while to run). If issues are detected, run ping-cluster against the node(s) with issue.

5.6) Check cluster-switch output::>


cluster-switch show -type cluster-network


THE END