Get All Modified Files in a Clustered ONTAP Volume using API

Introduction

Get-ModifiedFiles.ps1 is a PowerShell tool to find all the modified files in a Clustered ONTAP volume in the last X hours. Basically it works by recursively doing a Read-NcDirectory within all the folders in the volume. The script can either be run interactively like so:

PS C:\> .\Get-ModifiedFiles.ps1

Or non-interactively:

PS C:\ > .\Get-ModifiedFiles.ps1 -Cluster CLUSTERNAME -Username CDOTLOGIN -Password PASSWORD -Vserver SVMNAME -Volume VOLUME -Hours X

WARNING: It is not recommended to run this on production CDOT clusters since it can generate substantial CPU load whilst it is running!

The Script

Note: Tabs have been removed to display the script better in blogger.

# Get-ModifiedFiles.ps1
# A program to find the files modified in a specific NetApp Clustered ONTAP Volume, in the last X hours!

Param(
[String]$Cluster,
[String]$Username,
[String]$Password,
[String]$Vserver,
[String]$Volume,
[Int16]$Hours
)

"";"<<< Get Files Modified in the Last X Hours in a Volume >>>";""

# CHECK FOR DATA ONTAP POWERSHELL TOOLKIT

If(!(Get-Module DataONTAP)){Import-Module DataONTAP -ErrorAction SilentlyContinue}
If(!(Get-Module DataONTAP)){"Cannot find Data ONTAP PowerShell Toolkit - exiting!";"";exit}

# ACQUIRE DETAILS FOR THE SCAN

$SecurePass = $null
If($Password)    { $SecurePass = $Password | ConvertTo-SecureString -AsPlainText -Force }

If(!$Cluster -or !$Username -or !$SecurePass -or !$Vserver){
">>> Enter Connection Details ";""
If(!$Cluster)    { $Cluster    = Read-Host "Cluster " }
If(!$Username)   { $Username   = Read-Host "Username" }
If(!$SecurePass) { $SecurePass = Read-Host "Password" -AsSecureString}
If(!$Vserver)    { $Vserver    = Read-Host "Vserver " }
""
}

$Credential = New-Object System.Management.Automation.PsCredential($Username,$SecurePass)
$Connection = $null
$Connection = Connect-NcController -Name $Cluster -Credential $Credential -ErrorAction SilentlyContinue
If(!$Connection){"Unable to connect to $Cluster - exiting!";"";exit}
$Connection = Get-NcVserver $Vserver
If(!$Connection){"Unable to connect to $Vserver - exiting!";"";exit}
$Global:CurrentNcController.Vserver = $Vserver

# ACQUIRE VOLUME DETAILS

If(!$Volume){
">>> List of Volumes in $Vserver ";""
$Attributes = Get-NcVol -Template
$Query      = Get-NcVol -Template
Initialize-NcObjectProperty -object $Query -name VolumeStateAttributes
$Query.VolumeStateAttributes.IsVserverRoot = ""
$VolumesList = (Get-NcVol -Attributes $Attributes -Query $Query).Name
$VolumesList
""
$Volume   = Read-Host "Volume  "
}

# CHOOSE HOW MANY HOURS BACK TO CHECK

IF(!$Hours){$Hours = Read-Host "Hours   ";""}

# GETTING TIMESTAMP FROM A NODE (NODE 1)

$Global:CurrentNcController.Vserver = $null
[int64]$Timestamp  = 0
$Attributes        = Get-NcNode -Template
$Nodes             = (Get-NcNode -Attributes $Attributes).Node
$Timestamp         = (Get-NcTime -Node ($Nodes[0])).localtime
$AdjustedTimestamp = $Timestamp - ($hours * 60 * 60)
$Global:CurrentNcController.Vserver = $Vserver

# STARTING

$StartDate = date
">>> START TIME = $StartDate";""

# SCAN THROUGH ALL FOLDERS IN THE VOLUME

[System.Array]$Global:ModifiedFiles = @()
[int64]$Global:FolderCount = 0
[int64]$Global:FileCount = 0

$Attributes = Read-NcDirectory -Template
$Attributes.Empty = ""
$Attributes.Type  = ""
$Attributes.Name  = ""
$Attributes.ModifiedTimeStamp = ""

$ReadPath = "/vol/$Volume"

Function GetDirInfoRecursive {
Param([String]$PathToReadRecursive)     
Write-Host "." -ForegroundColor Cyan -NoNewLine
$GetDirInfo = Read-NcDirectory $PathToReadRecursive -Attributes $Attributes
Foreach ($line in $GetDirInfo){
If($line.Type -eq "directory"){
If(($line.Name -ne ".") -and ($line.Name -ne "..") -and ($line.Name -ne ".snapshot")) {
$Global:FolderCount ++
If ($line.Empty -ne "False"){
$NewPathToRead = $PathToReadRecursive + "/" + $line.Name
GetDirInfoRecursive $NewPathToRead             
}
}                   
} elseif($line.Type -eq "file"){
$Global:FileCount ++
If($line.ModifiedTimeStamp -gt $AdjustedTimestamp){
$FileDetails = ($line.Name) + " @ " + $PathToReadRecursive
$Global:ModifiedFiles += $FileDetails
}
}
}
}

GetDirInfoRecursive $ReadPath

# FINISHING

$FinishDate = date
"";"";">>> FINISH TIME = $FinishDate";""

# OUTPUT TO SCREEN

">>> OUTPUT <<<";""
"Files modified $Hours hours before $StartDate :";""
$Global:ModifiedFiles
"";"Processed " + $Global:FileCount + " files and " + $Global:FolderCount + " folders."
"Started at $StartDate"
"Finished at $FinishDate";""

# OUTPUT TO NOTEPAD

$UsersProfilePath = $env:USERPROFILE
$Date = Get-Date -uformat "%Y%m%d%H%M"
$OutputFile = $UsersProfilePath + "\ModifiedFiles_" + $Cluster + "_" + $Vserver + "_" + $date + ".txt"
$Global:ModifiedFiles | Out-File $OutputFile
Notepad $OutputFile

# END

Example Outputs

Example 1: Interactive
Example 2: Non-Interactive

Comments

  1. This is great! I do have a question though- which would cause a greater load: this script doing a recursive read-ncdirectory, or a similar script doing a get-childitem?

    ReplyDelete
  2. Hi Norm,

    Good question!

    I'd think get-childitem would cause greater load since you've got to go over CIFS. There's advantages to using APIs direct to the fileserver.

    I did improve this script, the following might be a better version (does things slightly differently, but same recursive function.)

    http://www.cosonok.com/2017/02/treesize-for-clustered-ontap-get.html

    Cheers, VC

    ReplyDelete
  3. Thanks, VC! That was my feeling as well.

    And thank you for the link to your updated script... It's really clean!

    ReplyDelete

Post a Comment