7CGT: 7 to C Gran Turismo

This is a blog post about a bespoke tool that helped perform NetApp 7-Mode to Clustered ONTAP migrations. If you’ve looking for Gran Turismo content, you’ve come to the wrong place!

Image: 7CGT Logo (created especially for this blog post)

I wasn’t going to blog about this very bespoke 7 to C migration tool, yet now seems the right time (and I’m writing this in my spare time on a Sunday too). Here’s the story, the ‘why I’m revisiting it’ and more...

If you’re wondering about the name, I made the name up myself - yes, I’m a bit of a nerd. It had originally been called “7 to C Clustershell Commands Generation Tool”, which fairly well described what it did - later versions generated PowerShell scripts to automate the application of the many many lines of CIFS shares and Exports.

The Story

Back in 2014, I was involved in a major 7 to C transition project. I cannot say who it was for. It did involve 2 x 6-node production Clustered ONTAP clusters, and the transitioning of some 24 p/vFilers down to 2 production SVMs (18 to 1 for the first production cluster, and 6 to 1 for the second production cluster).

At the time we started the project, 7MTT was at version 1.2 and wasn’t really up to the job we wanted to do (remember, this was 2014). In fairness, 7MTT is designed to be a transition tool, not a restructuring/consolidation tool; there were also certain features 7MTT didn’t have at the time which it does now. So, we ended up taking the path we did - the bespoke tool path (we didn’t take this path lightly). 7MTT these days (currently at version 3.2) is an outstanding tool, and I’m certain if we’d had this version 3 years ago, 7CGT would not have been born.

The project was successfully completed towards the end of 2015 (October time). 7CGT created all the commands required for the 7 to C migration. That was 24 p/vFilers down to 2 production SVMs, 320+ volumes, 4000+ CIFS shares, 5600+ CIFS share ACLs, 1300+ unix-win name-mappings, 1200+ win-unix name-mappings, 600+ export policies, 4700+ export policy rules, 1450+ UNIX users, 180+ UNIX groups, and  1700+ quota policy rules - and all done using 7CGT.

Note: It’s always been easy to transition volumes from 7 to C, since it just needs a special type of snapmirror. Transitioning the configuration from 7 to C is trickier!

Why the Revisit

It’s now April 2017, 1.5 years since the aforementioned project was finished, and 1.5 years since 7CGT last got used in anger (at least that I know of.) Now an opportunity has arisen to revisit some of what 7CGT did, and see if parts might be useful in an upcoming project (re-worked for a new version of ONTAP.)

What did 7CGT do?

7CGT had 3 independent modes of operation:

- Transition Setup Commands Mode

- Configurations and Cutover Commands Mode

- Post Cutover SnapVault Commands Mode

Image: 7CGT Modes of Operation (colorful DOS graphics!)

It had an offline mode (didn’t need to be on the corporate network, just needed config files), or an online mode where it would get all the data it needed from 7-Mode and cDOT systems (reading config files etcetera.)

It did expect pre-requisites such as:

- Data SVM already built to a known and approved standard (and p/vFilers being transitioned had a suitable known and approved build standard)

- Cron schedules created for SnapMirrors

- Snapshot Policies created

- Default ‘Read Only’ and ‘No Access’ export policies created

- SnapVault policy (for Post Cutover mode)

Then there were the 7CGT control files, used to setup the 7CGT processing run:

- ‘Volumes To Migrate’ file. Containing lines like:

PFILER,VFILER(or vFiler0),VOLNAME

- ‘Volume Name Mapping’ file (optional). Containing lines like:

PFILER,VOLNAME,CDOT-VOLNAME

- ‘Data Copy IPs’ file. Containing lines like:

7GPrimarySystemName,DataCopyIP,7GSecondarySystemName,DataCopyIP

The environment was CIFS and NFS, and the following 7-Mode config files had to be collected for each p/vFiler with volumes to be processed (remember, we had company standard source 7-Mode and destination cDOT builds, so appropriate SVM settings were covered already):

- cifsconfig_share.cfg

- exports*

- groups*

- passwd*

- usermap.cfg*

- quotas**

*Only required if processing NFS configurations.

** Only required if processing Quota rules.

7CGT would process these 7-Mode config files into Clustered ONTAP clustershell commands, which you could them review, modify if required, and apply. 7CGT also generated a script of PowerShell commands, and batch file - to run the PowerShell - which you could simply double-click to apply 100s of CIFS shares or export rules (that’s if copying and pasting 1000s of lines via SSH isn’t your idea of fun.)

Other 7CGT config files that needed to be setup - depending on the 7CGT mode and whether running offline or online (or a mixture of online/offline - perhaps online to cDOT but offline access to 7-Mode config files):

- ‘Transition Setup’ file

- ‘SnapVault Setup’ file

- ‘SID to Name Mapping’ file

- ‘Not a Qtree Override’ file (i.e. where with /vol/VOLNAME/FOLDER, FOLDER is not a qtree)

Files generated from the Clustered Data ONTAP systems:

- ‘CDOT Shares’ file

- ‘CDOT Unix Groups’ file

- ‘CDOT Unix Users’ file

- ‘CDOT Name Mapping’ file

Once all the required data to run the tool had been collected (reading the above it seems a hassle, but it really wasn’t that bad, especially since a lot could be gathered by the tool itself - that’s if it had API access to the 7-Mode and cDOT Systems, otherwise it would work offine), then it was just a case of running the tool to generate all the Clustershell (or PowerShell) commands required for an end-to-end 7 to C migration. There were of course various prompts for further information, all of which could be overridden by an INI file.

Could 7CGT be Reused/Improved?

My first resort would always be 7MTT (now version 3.2), and this will serve 99.9% of transition scenarios excellently.

7CGT was designed for transitions to Clustered ONTAP 8.2.x, so some updating would be required for the current latest ONTAP GA version - 9.1.

In all honesty, 7CGT wasn’t a great bit of code - roughly 2000 lines or so, and could have been written better. If I was to revisit the tool, I’d have written it in a more modular fashion i.e.:

- CIFS Shares 7 to C

- NFS Exports 7 to C (this did get written see here - but I wrote a better version that I’ve yet to blog)

- UNIX groups 7 to C

- UNIX users 7 to C

- Usermap 7 to C

- Quotas 7 to C (this did get written see here)

And write it so that it is WFA-able too (would need to save a report somewhere though - WFA is not a reporting tool)!