Search-NcFileSystem & Search-NaFileSystem

Introduction

In the next two posts, I’ll post the PowerShell code for:

Search-NcFileSystem.ps1

Search-NaFileSystem.ps1

The first one (Nc) is for Clustered Data ONTAP. And the second (Na) is for 7-Mode. These are PowerShell tools for scanning a filesystem (volume) for particular file/files and/or folder/folders, or you can just get a list of all the folders and files on that volume.

They are straightforward to run and can be parameterized on the CLI as desired.

The code is adapted from the code in this post from the start of the year:

Get All Modified Files in a Clustered ONTAP Volume using API

FAQ

Q: Can I scan RO volumes (the SnapMirror-ed DR) without breaking the mirror?

A: YES

Q: What impact does running this have on the system?

A: It will consume a significant amount of CPU and disk resources; hence it is recommended to run this on DR or test systems.

IMPORTANT: Please do not run these tools against production volumes. If you do then please at least keep an eye on CPU and disk utilization!

Q: Does this need the DR volume to be mounted (for cDOT)?

A: NO. The tool uses APIs so it’s more efficient than file scanning tools that need to use NFS/CIFS access protocols - this tool does not even need NFS or CIFS to be running.

Performance

All these tests were run on a VSIM (either cDOT or 7-Mode) with 2 vCPUs, 2.9GB memory, and VMDK’s on SSD.

Test 1: cDOT 8.2.3 2-Node Cluster - first run

Processed 1 volume with 10’411 files and 1767 folders

Time = 157 seconds

Test 2: cDOT 8.2.3 2-Node Cluster - second run

Processed 1 volume with 10’411 files and 1767 folders

Time = 84 seconds

Note: The reason why it was faster the second time is probably because more of the file/folder information data was in memory.

Test 3: 7-Mode 8.1.2P4 - first run

Processed 1 volume with 10’361 files and 1729 folders

Time = 728 seconds

Note: The APIs are improved and more efficient in cDOT, hence the longer running time on 7-Mode is no surprise!

Test 4: 7-Mode 8.1.2P4 - second run

Processed 1 volume with 10’411 files and 1767 folders

Time = 716 seconds

Note: Interesting that the second run wasn’t much quicker.

Performance Estimation

On the basis of very little performance analysis (apologies, simply don’t have the time to do a thorough analysis) and on the hardware used in this lab:

Search-NcFileSystem (cDOT) can process:

10’000 files in 100 seconds

1 Million files in less than 3 hours.

Search-NaFileSystem (7-Mode) can process:

10’000 files in 700 seconds

1 Million files in ~ 20 hours