Friday, 29 January 2021

[ONTAP 9.7] Steps to add 2nd NS224 Shelf to NetApp AFF A400

Setting the Scenario
 
You have a NetApp AFF A400 with 1 external NS224 shelf.
You already have an IO card in slot 5 (say for serving ethernet or fibre channel data).
You need to add a 2nd NS224 shelf.
 
High-Level Overview
 
The NS224 with NSM100 shelf modules has a max stack size of 1 (see HWU - hwu.netapp.com.) This means that in order to add a 2nd NS224 shelf to the A400, you need an X1148A NVMe Storage card (assuming you don’t have one already) and this must go in slot 5 following the ‘Priority Slot Assignment’ on HWU.
 
Starting point is one NS224 shelf cabled as:
 
Shelf|NSM|Shelf|Node |Node
     |   |Port |Port |
-----+---+-----+-----+----
01   | A | e0a | e0c | A
01   | B | e0a | e0c | B
01   | B | e0b | e0d | A
01   | A | e0b | e0d | B
 
Where:
- NSM A and Node A are the topmost NSM/Node.
- NSM B and Node B are the bottommost NSM/Node.
 
Image: NetApp AFF A400 cabled with one NS224 shelf

 
The process to add the 2nd shelf is:
 
1) Take over Node A
2) [Node A] Remove from the chassis
3) [Node A] Swap IO card in slot 5 into slot 1 (check HWU)
4) [Node A] Insert X1148A NVMe Storage card into slot 5
5) [Node A] Unplug e0d storage connection
6) [Node A] Boot and perform partial giveback
7) [Node A] Configure slot 5 storage ports
8) [Node A] Old e0d storage connection connects to e5b
9) [Node A] Perform necessary re-configurations for slot 1 IO card
10) [Node A] Complete giveback
11) Take over Node B
12) [Node B] Remove from the chassis
13) [Node B] Swap IO card in slot 5 into slot 1 (check HWU)
14) [Node B] Insert X1148A NVMe Storage card into slot 5
15) [Node B] Unplug e0d storage connection
16) [Node B] Boot and perform partial giveback
17) [Node B] Configure slot 5 storage ports
18) [Node B] Old e0d storage connection connects to e5b
19) [Node B] Perform necessary re-configurations for slot 1 IO card
20) [Node B] Complete giveback
21) [NS224 Shelf 2] Set shelf ID
22) [NS224 Shelf 2] Hot-add using node ports e5a (path A) and e0d (path B)
 
Finishing point is 2 NS224 shelves, cabled as:
 
Shelf|NSM|Shelf|Node |Node
     |   |Port |Port |
-----+---+-----+-----+----
01   | A | e0a | e0c | A
01   | B | e0a | e0c | B
01   | B | e0b | e5b | A
01   | A | e0b | e5b | B
02   | A | e0a | e5a | A
02   | B | e0a | e5a | B
02   | B | e0b | e0d | A
02   | A | e0b | e0d | B
 
Note: e5b (2) is to the left, and e5a (1) is to the right in the below diagram.
 
Image: NetApp AFF A400 cabled with one NS224 shelf


End-to-end, this process shouldn’t take longer than 4 hours (perhaps even less than 2 hours.)
 
Low-Level Commands
 
The low-level commands used in the operation will be bespoke to the particular system (for instance: the card originally in slot 5 might home SAN LIFs, or it might home NAS LIFs.) The below is a rough guide.
 
1) Take over Node A
 
Send AutoSupport and disable auto-giveback:
 
timeout
timeout modify -timeout 0
autosupport invoke -node * -type all -message "MAINT=4h Moving IO cards 1 to 5 and adding a shelf"
storage failover show -node * -fields auto-giveback
storage failover modify -node * -auto-giveback false
storage failover show -node * -fields auto-giveback
 
Card 5 had NAS LIFs: Temporarily rehome Node A’s LIFs to Node B:
 
net int modify -vserver VSERVER -lif LIFNAME -home-node NODE-B -home-port PORT
net int revert -vserver VSERVER -lif LIFNAME
 
Card 5 had SAN LIFs: Leave for now.
 
Takeover Node A:
 
storage failover takeover -ofnode NODE-A
 
When the output says ‘Hit [Enter] to boot immediately, or any other key for command prompt.’ press any other key. When the system is at the LOADER> prompt it is safe to continue.
 
2) [Node A] Remove from the chassis
3) [Node A] Swap IO card in slot 5 into slot 1 (check HWU)
4) [Node A] Insert X1148A NVMe Storage card into slot 5
5) [Node A] Unplug e0d storage connection
6) [Node A] Boot and perform partial giveback
 
From Node B, perform a partial giveback:
 
storage failover giveback -ofnode NODE-A -only-cfo-aggregates true
storage failover show-giveback
 
7) [Node A] Configure slot 5 storage ports
 
storage port enable –node NODE-A –port PORT
 
8) [Node A] Old e0d storage connection connects to e5b
9) [Node A] Perform necessary re-configurations for slot 1 IO card
 
For NAS LIFs: Rebuild the IFGRPs/Ports/VLANs and perform clean up.
 
For SAN LIFs:
 
net int modify -vserver VSERVER -lif LIFNAME -status-admin down
net int modify -vserver VSERVER -lif LIFNAME -home-port PORT -home-node NODE-A
net int modify -vserver VSERVER -lif LIFNAME -status-admin up
set adv
net port modify -node NODE-A -port PORT -up-admin down
net port modify -node NODE-A -port PORT -up-admin up
 
10) [Node A] Complete giveback
 
storage failover giveback -ofnode NODE-A
storage failover show-giveback
 
For NAS LIFs: Rehome back to NODE-A:
 
net int modify -vserver VSERVER -lif LIFNAME -home-node NODE-B -home-port PORT
net int revert -vserver VSERVER -lif LIFNAME
 
11) Take over Node B
 
Card 5 had NAS LIFs: Temporarily rehome Node B’s LIFs to Node A:
 
net int modify -vserver VSERVER -lif LIFNAME -home-node NODE-A -home-port PORT
net int revert -vserver VSERVER -lif LIFNAME
 
Card 5 had SAN LIFs: Leave for now.
 
Takeover Node B:
 
storage failover takeover -ofnode NODE-B
 
When the output says ‘Hit [Enter] to boot immediately, or any other key for command prompt.’ press any other key. When the system is at the LOADER> prompt it is safe to continue.
 
12) [Node B] Remove from the chassis
13) [Node B] Swap IO card in slot 5 into slot 1 (check HWU)
14) [Node B] Insert X1148A NVMe Storage card into slot 5
15) [Node B] Unplug e0d storage connection
16) [Node B] Boot and perform partial giveback
 
From Node A, perform a partial giveback:
 
storage failover giveback -ofnode NODE-B -only-cfo-aggregates true
storage failover show-giveback
 
17) [Node B] Configure slot 5 storage ports
 
storage port enable –node NODE-B –port PORT
 
18) [Node B] Old e0d storage connection connects to e5b
19) [Node B] Perform necessary re-configurations for slot 1 IO card
 
For NAS LIFs: Rebuild the IFGRPs/Ports/VLANs and perform clean up.
 
For SAN LIFs:
 
net int modify -vserver VSERVER -lif LIFNAME -status-admin down
net int modify -vserver VSERVER -lif LIFNAME -home-port PORT -home-node NODE-B
net int modify -vserver VSERVER -lif LIFNAME -status-admin up
set adv
net port modify -node NODE-B -port PORT -up-admin down
net port modify -node NODE-B -port PORT -up-admin up
 
20) [Node B] Complete giveback
 
storage failover giveback -ofnode NODE-A
storage failover show-giveback
 
For NAS LIFs: Rehome back to NODE-B:
 
net int modify -vserver VSERVER -lif LIFNAME -home-node NODE-B -home-port PORT
net int revert -vserver VSERVER -lif LIFNAME
 
21) [NS224 Shelf 2] Set shelf ID
22) [NS224 Shelf 2] Hot-add using node ports e5a (path A) and e0d (path B)
 
node run local storage show disk -p
node run !local storage show disk -p
storage failover modify -node * -auto-giveback true
storage failover show -node * -fields auto-giveback
autosupport invoke -node * -type all -message "MAINT=END"
timeout modify -timeout 30
 
THE END
 
Also see:
https://examcramnotes.blogspot.com/2021/01/netapp-aff-a400-max-ns224-shelves-in.html

Thursday, 28 January 2021

Tech Roundup - January 2021

I’ve not managed to do one of these ‘Tech Roundups’ since June 2020, and I wasn’t going to sift through 7 months of saved links and stuff, so this is a fresh start using the last 2 months data. Also, I’ve tried to be a bit more discerning with what I link, which is why I’ve put a section at the bottom of the post entitled “Go to Places for Regular Tech Updates”. My intention is to do these posts bimonthly going forward.
 
Azure Stack HCI
 
Azure Stack HCI solution overview
https://docs.microsoft.com/en-us/azure-stack/hci/overview
“This is a dedicated hyperconverged operating system based on Windows Server Core and managed by Windows Admin Center, which is designed to make it easier to deploy Hyper-V with Storage Spaces Direct (S2D). This is for on-premises Hyper-V! What makes the offering different from regular Hyper-V though, is that it is delivered as an Azure hybrid service: it comes with a monthly subscription cost and it is connected to Azure.”
 
Video for the link above:
Discover Azure Stack HCI
https://www.youtube.com/watch?v=fw8RVqo9dcs [3 mins]
 

 
CentOS
 
CentOS is gone—but RHEL is now free for up to 16 production servers
https://arstechnica.com/gadgets/2021/01/centos-is-gone-but-rhel-is-now-free-for-up-to-16-production-servers/
 
Also see:
https://developers.redhat.com/articles/faqs-no-cost-red-hat-enterprise-linux#
https://blog.centos.org/2020/12/future-is-centos-stream/
https://www.zdnet.com/google-amp/article/why-red-hat-dumped-centos-for-centos-stream/
 
And:
Rocky Linux promises to become the new "good old CentOS"!
https://github.com/rocky-linux/rocky
 
GitHub
 
Github's Fork & Pull Workflow for Git Beginners
https://reflectoring.io/github-fork-and-pull/
 
GitHub & NetApp & Ansible
 
Where NetApp Ansible playbooks will be posted and shared from (and more):
https://github.com/NetApp-Automation
 
Also see:
https://netapp.io/2018/10/08/getting-started-with-netapp-and-ansible-install-ansible/
 
Kubernetes
 
The Kubernetes Aquarium
https://medium.com/@AnneLoVerso/the-kubernetes-aquarium-6a3d1d7a2afd
 
Don't Panic: Kubernetes and Docker
https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/
 
NetApp
 
Backup got better - File level restore on NetApp Cloud Backup Service (CBS) DEMO
https://www.youtube.com/watch?v=ROAY6gPL9N0 [3 mins]
 
NetApp IT Systems Engineer demos how he uses Trident, OpenShift and Ansible
https://netappit.com/news/netapp-it-systems-engineer-demos-how-he-uses-trident-openshift-and-ansible/
https://www.youtube.com/watch?v=jgpg7m1JbCU [12 mins]
 
How to use Ansible to provide S3 as a Service with StorageGRID
https://netappit.com/news/s3-as-a-service
https://www.youtube.com/watch?v=BOv0lQonLts [12 mins]
 
More than 25 DevOps and Automation sessions on demand from NetApp INSIGHT 2020
https://blog.netapp.com/devops-automation-sessions-on-demand-insight-2020
 
Useful:
[NVMe-oF] How to Setup NVMe-oF with VMware and NetApp
[ONTAP 9.6 & 9.7] Checking Storage Efficiencies on NetApp AFF
ONTAP: Easily Identify Remaining Headroom
 
NetApp Data Science Toolkit
 
https://github.com/NetApp/netapp-data-science-toolkit
The NetApp Data Science Toolkit is a Python library that makes it simple for data scientists and data engineers to perform various data management tasks, such as provisioning a new data volume, near-instantaneously cloning a data volume, and near-instantaneously snapshotting a data volume for traceability/baselining.
 
"Provision a New Data Volume with the NetApp Data Science Toolkit"
https://www.youtube.com/watch?v=Wt1b3yYZFOQ
https://tv.netapp.com/detail/video/6221336271001
 
"Near-Instantaneously Clone a Data Volume with the NetApp Data Science Toolkit"
https://www.youtube.com/watch?v=0NitnWodvhM
https://tv.netapp.com/detail/video/6221334320001
 
NetApp ONTAP 9.8 REST API: Mapping ONTAPI to REST API & More!
 
ONTAP REST API Resources:
NetApp Developer Network  
 
Documentation:
Release Notes
ONTAP 9.8 API Reference
ONTAP 9.8 Python Client Library
ONTAP 9.x REST API documentation
 
Transition to REST from ONTAPI:
Mapping ONTAPI to ONTAP 9.8 REST API
 
Blogs:
Getting Started with ONTAP REST API Python Client Library – Part 1 , Part 2 & Part 3
Using the private CLI passthrough with ONTAP REST API
ONTAP REST API – Automate notifications of high-severity events
 
Sample scripts:
ONTAP REST API python scripts GitHub 
 
NetApp Plugin for Symantec NetBackup
 
Upcoming removal of support for ‘NetApp Plugin for Symantec NetBackup’:
It is being revised into separate 7-mode and C-Mode versions:
 
Product: NetApp Plug-in 1.x for Symantec NetBackup
ONTAP: ONTAP Operating in 7-Mode
EOES: 31-Dec-2020
End of Limited Support: 30-Apr-2021
End of Self-Service Support: 30-Apr-2024
 
Product: NetApp Plug-in 2.x for Symantec NetBackup
ONTAP: Clustered Data ONTAP/ONTAP 9
EOES: 31-Dec-2020
End of Limited Support: 31-Dec-2022
End of Self-Service Support: 31-Dec-2025
 
OpenShift
 
I’m So Sorry OpenShift, I’ve Taken You for Granted
https://medium.com/swlh/im-so-sorry-openshift-i-ve-taken-you-for-granted-f36fb47ea4d9
A 5-minute read that explains user experience between Openshift & (Vanilla) Kubernetes (OCP comes with a whole framework on top of Kubernetes, with valuable features like RBAC.) The post describes how to create an app from scratch, and the process to deploy and access it with both platforms.
 
Storage Industry News
 
8 Emerging Data Storage Trends To Watch In 2021
https://www.crn.com/slide-shows/storage/8-emerging-data-storage-trends-to-watch-in-2021
Storage In 2021 And Beyond: Less Focus on Tech, More Focus on Data Value:
1) The Line Between On-Prem And Cloud Will Be Erased
2) Software-Defined to Take the Wind from Storage Hardware Sales
3) No More Vendor “A” vs. Vendor “B”
4) ‘Speeds and Feeds’ Morphs To ‘Needs’
5) Storage Will Be Increasingly Container-Native
6) Storage March to The Edge Accelerates
7) “Security-First” Mantra to Define Storage
8) Storage Will Get Smarter
 
Go to Places for Regular Tech Updates
 
https://tv.netapp.com
https://netappit.com/news/
https://soundcloud.com/techontap_podcast
https://netapp.io/
https://cloud.netapp.com/blog
https://blog.netapp.com/
https://spot.io/blog/

Monday, 25 January 2021

The Home Lab is Dead! Long Live the Home Lab!

My Home Lab Workstation is dead as a dodo now. It won’t boot into Windows. Tried multiple attempts to boot. Tried reset and reinstall. Just boots so far then gets stuck (the spinning dots stop spinning.) It had been a bit temperamental booting for a while - would need multiple boots to get it to boot - but then once booted it ran fine.
 
I had to check back to see when I got this PC, and it would have been late 2012 (http://www.cosonok.com/2012/10/new-home-lab-workstation-64gb-ram-125tb.html)! The PC has lasted well over 8 years which is pretty good going - I did not realize I had had it so long - and it’s not had an easy life with my VMware Workstation labs and also some video editing. My O/S drive is an SSD drive, and this is where I think the fault lies (I’ve done little troubleshooting, this is just a hunch.) Other than this current issue, all I’ve had to do to the PC is replace the BIOS battery once, and the CPU fan once. And it is still a very powerful home PC (64GB RAM and 8 * 3.6GHz cores.)
 
The O/S drive gets heavily used, and reading this article 'Lifespan of Solid-State Drives' and ‘How long does an SSD last?’ it says “Current estimates put the age limit for SSDs around 10 years, though the average SSD lifespan is shorter.” It is possible my O/S SSD has bit the dust. Fortunately, when I bought the PC I went a bit crazy and had 3 SSDs (1 * 256GB for O/S, 2 * 512GB for lab VMs), so my plan is to unplug my O/S SSD, and reinstall Windows on one of the other SSDs, and see how it goes.
 
I did consider installing CentOS or another flavour of Linux, but for now let’s stick with Windows (there is an HDD data drive in the PC I would like to regain access to.) Interestingly, looking at CentOS versions, I discovered Version 7 went end of full support on August 6, 2019 - see: https://access.redhat.com/support/policy/updates/errata/#Life_Cycle_Dates - something I didn’t know.
 
Let us see if we can get a bit more life out of this PC without spending any money. If I’m correct about the O/S SSD being knackered, I’ll consider investing in a new SSD disk for the future!
 
Re-Installing Windows 10 on a New/Different SSD
 
A web search directed me to the ‘Download Windows 10’ page and the tool to ‘Create Windows 10 installation media’ (MediaCreationTool20H2.exe):
https://www.microsoft.com/en-gb/software-download/windows10
 
Image: Create Windows 10 installation media ‘Download tool now’


Run the exe and follow the prompts (from a different machine, since my machine is bust).
At the ‘What do you want to do?’ prompt, choose “Create installation media (USB flash drive, DVD, or ISO file) for another PC”.
 
Note: I made the mistake of initially picking DVD, alas you need an 8.5GB double-layer DVD to contain the image, and I only had single-layer 4.7GB to hand, so re-ran the process with a USB flash drive.
 
Image: Create Windows Installation Media for Another PC


Then you just need to boot from the installation media and follow the prompts. When it asks you to ‘Activate Windows’ choose ‘I don’t have a product key’ and it should reconnect with your old key later.
 
This Microsoft blog had a good complete walkthrough:
https://answers.microsoft.com/en-us/windows/forum/windows_10-windows_install-winpc/clean-install-of-windows-10-os-on-new-ssd-best/9186ae98-b2b4-4b7a-b3ff-0b053ce2d1c9
 
The lab is back! Next weekend (time permitting) I’ll need to reload softwares and get my lab VMs up and running.

Sunday, 24 January 2021

NetApp E-Series E2760 8.20 to 8.40 Upgrade Experience

Been a while since I last had to upgrade an E-Series System. The E2760 running SANtricity 8.20 doesn’t use the new System Manager, so the first thing you’ll need to do is upgrade the E-Series SANtricity Storage Manager client.
 
Before you can start the E-Series controller upgrades:

  1. The system needs to be optimal.
  2.  No operations should be in progress.
  3. Go through the event log checking for issues. Clear down the log when you’re satisfied there’s nothing that needs resolving before the upgrade.
Image: Things to Check Prior to Upgrading Your E2760

 


 
There are 5 things to upgrade, and in this order (not including Host specific software here):
  • 1) Upgrade ‘SANtricity Storage Manager’ client
    • ~45 minutes - needs a server reboot
  • 2&3) Upgrade the ‘Controller Firmware’ and ‘Controller NVSRAM...’ (can do at the same time)
    • ~45 minutes
  • 4) Upgrade ‘ESM Firmware...’
    • ~45 minutes
  • 5) Upgrade ‘Disk Firmware...’
    • Can leave this running after everything else is done. It will update firmware one disk per disk-pool at a time.
 Image: Things to Upgrade in SANtricity Array Manager



Things to Download:

 
1) Download E-Series SANtricity Storage Manager 11.53.0X00.0013 from:
https://mysupport.netapp.com/site/products/all/details/eseries-santricity/downloads-tab/download/62736/11.53.0X00.0013/downloads


2) Download 8.40.60.01 firmware for E2760
https://mysupport.netapp.com/site/products/all/details/eseries-santricityos/downloads-tab/download/62735/8.40.60.01/downloads

Download SANtricity OS Software

 
Download SANtricity NVSRAM (Dual Controller)

 
Download IOM Firmware for SAS2 Enclosures

 
3) Download E-Series Disk Firmware
https://mysupport.netapp.com/NOW/download/tools/diskfw_eseries/instructions.html?zp=all.zip&gz=all.tar.gz

Friday, 22 January 2021

[ONTAP 9.7+] Roles Created by NetApp’s VSC 9.7.1

From this link:
VSC, VASA Provider, SRA 9.7: Configuring User Roles and Privileges
 
The documentation tells us to download the ‘ONTAP Privileges’ file from:
https://{virtual_appliance_IP}:9083/vsc/config/VSC_ONTAP_User_Privileges.zip
 
This zip contains a file called VSC_user_roles.json. You upload the JSON file via the ONTAP 9.7+ System Manager -
Cluster > Settings > Users and Roles > Add User
- selecting ‘Virtualization products’ and choose ‘Product Capability’ which gives the choice -

  • VSC 9.7
  • VSC and VASA Provider 9.7
  • VSC and SRA 9.7
  • VSC, VASA Provider and SRA 9.7
- Specify  a user name (new user who will be assigned the role), password for the user, pick your privileges -
  • Discovery: Allows discovery of all connected storage controllers.
  • Create Storage: Allows creation of volumes and LUNs.
  • Modify Storage: Allows resizing and deduplicating of storage.
  • Destroy Storage: Allows destruction of volumes and LUNs.
  • NAS/SAN Role: Allows discovery of all connected storage controllers, only on VMware SRM environment.
- and finally click Add.
 
The above allows for a number of different roles (NAS/SAN Role only appears when SRA 9.7 is selected.)
 
Image: Add User > Virtualization Products > VSC_user_roles.json + Product Capability

 
Image: ONTAP 9.7 > Virtualization Products > Privileges

 
ONTAP Privileges
 
There’s too many different roles to document in this blog, so I’ll document just the one I’m particularly interested in, which is this one:
 
Virtualization Products:
Product = VSC, VASA Provider and SRA
Product Capability = VSC and VASA Provider 9.7
Privileges = Discovery + Create + Modify + Destroy
 
I’m not interested in SRA 9.7 product capability for this scenario. I want all the privileges, and later on I will attempt to modify the privileges to disable VMware Admins from creating/destroying flexvols (they need to be able to create/destroy LUNs), with an eye to giving them enough permission to do everything they need to do in order to manage VMs on VVOLs, just leave it to a storage admin to provision the flexvols for the VVOL datastores.
 
Note 1: “If VASA Provider is required for a particular storage controller, then the storage system must be added to VSC at the cluster level.” - source
Note 2: All the users are added with application = ontapi
 
These are the access and cmddirname specified by the role UnifiedVirtualApplianceVSC&VP9.7_Discovery_Create_Modify_Destroy:
 
ACCESS   : CMDDIRNAME
---------+-----------
none     : DEFAULT
readonly : cluster identity modify
readonly : cluster identity show
readonly : cluster modify
readonly : cluster peer show
readonly : cluster show
all      : job
readonly : job show-completed
all      : lun comment
all      : lun create
all      : lun delete
readonly : lun geometry
all      : lun igroup add
readonly : lun igroup create
readonly : lun igroup modify
all      : lun igroup set
readonly : lun igroup show
all      : lun mapping create
all      : lun mapping delete
all      : lun mapping show
all      : lun modify
all      : lun move
all      : lun offline
all      : lun online
all      : lun resize
all      : lun show
readonly : network fcp adapter modify
readonly : network fcp adapter show
readonly : network interface create
readonly : network interface delete
all      : network interface migrate
readonly : network interface modify
readonly : network interface show
readonly : network port delete
readonly : network port modify
readonly : network port show
all      : qos policy-group create
all      : qos policy-group modify
all      : qos policy-group show
readonly : security login create
readonly : security login delete
readonly : security login modify
readonly : security login role create
readonly : security login role delete
readonly : security login role modify
readonly : security login role show
readonly : security login role show-ontapi
all      : security login role show-user-capability
readonly : security login show
all      : set
readonly : snapmirror create
readonly : snapmirror list-destinations
readonly : snapmirror show
all      : snapmirror update-ls-set
readonly : storage aggregate create
readonly : storage aggregate modify
readonly : storage aggregate show
readonly : storage disk show
all      : storage failover modify
all      : storage failover show
readonly : system health alert modify
readonly : system health alert show
readonly : system health status show
readonly : system license delete
readonly : system license show
all      : system node autosupport invoke
readonly : system node modify
all      : system node run
readonly : system node show
readonly : version
all      : volume autosize
all      : volume clone create
all      : volume clone show
all      : volume create
all      : volume destroy
all      : volume efficiency modify
all      : volume efficiency off
all      : volume efficiency on
all      : volume efficiency show
all      : volume efficiency start
all      : volume efficiency stat
all      : volume efficiency stop
all      : volume file show-disk-usage
all      : volume modify
all      : volume offline
readonly : volume qtree create
readonly : volume qtree show
readonly : volume quota modify
readonly : volume quota report
readonly : volume quota show
all      : volume restrict
all      : volume show
all      : volume size
all      : volume snapshot create
all      : volume snapshot delete
all      : volume snapshot modify
all      : volume snapshot show
all      : volume unmount
readonly : vserver create
readonly : vserver export-policy create
readonly : vserver export-policy delete
all      : vserver export-policy rule create
all      : vserver export-policy rule delete
all      : vserver export-policy rule modify
all      : vserver export-policy rule setindex
all      : vserver export-policy rule show
readonly : vserver export-policy show
readonly : vserver fcp create
readonly : vserver fcp delete
readonly : vserver fcp initiator show
readonly : vserver fcp interface show
readonly : vserver fcp modify
readonly : vserver fcp show
readonly : vserver iscsi connection show
readonly : vserver iscsi create
readonly : vserver iscsi delete
all      : vserver iscsi interface accesslist add
readonly : vserver iscsi interface modify
readonly : vserver iscsi interface show
readonly : vserver iscsi modify
readonly : vserver iscsi session show
readonly : vserver iscsi show
readonly : vserver modify
readonly : vserver nfs create
readonly : vserver nfs delete
readonly : vserver nfs modify
readonly : vserver nfs show
all      : vserver nfs status
all      : vserver services name-service unix-group
all      : vserver services name-service unix-user
readonly : vserver show

NetApp ONTAP: Apply SACLs Using Vserver Security File-Directory

The following blog is an example of applying an everything SACL (CIFS audit policy) using the ‘vserver security file-directory’ command set.  Unfortunately, using ‘vserver security file-directory’, you cannot just add a SACL, you have to get the existing DACL, and then add the SACL and original DACL at the same time. In practice, setting NTFS SACLs from Windows is an easier approach (I’d recommend reading Justin Parisi's blog post here and especially the comments section.)
 
My ONTAP version here is 9.7.
 
Setting Up the Test Environment
 
My test structure is one volume with 2 folders:
vol1 > folder1 > folder2
And each folder has a text file in:
folder1file.txt and folder2file.txt
 
The permissions have been setup as so:
 
vol1:
  Everyone with ‘Read & execute’ access,
  applied to ‘folder, subfolder and files’,
  and Inheritance Disabled
 
folder1:
  ‘Domain Admins’ with ‘Full Control’,
  ‘Domain Users’ with ‘Read & execute’,
  applied to ‘folder, subfolder and files’,
  and Inheritance Disabled
 
folder2:
  ‘Domain Admins’ with ‘Full Control’,
  ‘Domain Users’ with ‘Modify’,
  applied to ‘folder, subfolder and files’,
  and Inheritance Disabled
 
Reviewing Current ACLs using ‘vserver security file-directory show’
 
The current ACLs as reviewed by ‘vserver security file-directory show’ are below:
 
Current /vol1 ACLs: 

ACLs: NTFS Security Descriptor
  Control:0x9504
  Owner:BUILTIN\Administrators
  Group:BUILTIN\Administrators
  DACL - ACEs
    ALLOW-Everyone-0x1200a9-OI|CI
 
Current /vol1/folder1 and folder1file.txt ACLs:
 
ACLs: NTFS Security Descriptor
  Control:0x9504
  Owner:BUILTIN\Administrators
  Group:DEMO\Domain Users
  DACL - ACEs
    ALLOW-DEMO\Domain Admins-0x1f01ff-OI|CI
    ALLOW-DEMO\Domain Users-0x1200a9-OI|CI
 
Current /vol1/folder1/folder2 and folder2file.txt ACLs:
 
ACLs: NTFS Security Descriptor
  Control:0x9504
  Owner:BUILTIN\Administrators
  Group:DEMO\Domain Users
  DACL - ACEs
    ALLOW-DEMO\Domain Admins-0x1f01ff-OI|CI
    ALLOW-DEMO\Domain Users-0x1301bf-OI|CI
 
Creating SACLs
 
We have 3 different permission sets, so we will need 3 NTFS Security Descriptors to rebuild the permissions above with a SACL.
 
Firstly, we create 3 NTFS Security Descriptors, each with our audit everyone and everything SACL (note, I don’t want inheritance, so not using the apply-to ‘sub-folders’.)
 
vserver security file-directory ntfs sacl add -ntfs-sd sdvol1 -access-type failure -account Everyone -vserver svm1 -rights full-control -apply-to this-folder,files
vserver security file-directory ntfs sacl add -ntfs-sd sdvol1 -access-type success -account Everyone -vserver svm1 -rights full-control -apply-to this-folder,files
 
vserver security file-directory ntfs sacl add -ntfs-sd sdfolder1 -access-type failure -account Everyone -vserver svm1 -rights full-control -apply-to this-folder,files
vserver security file-directory ntfs sacl add -ntfs-sd sdfolder1 -access-type success -account Everyone -vserver svm1 -rights full-control -apply-to this-folder,files
 
vserver security file-directory ntfs sacl add -ntfs-sd sdfolder2 -access-type failure -account Everyone -vserver svm1 -rights full-control -apply-to this-folder,files
vserver security file-directory ntfs sacl add -ntfs-sd sdfolder2 -access-type success -account Everyone -vserver svm1 -rights full-control -apply-to this-folder,files
 
And verify these are correct using the below (outputs not included):
 
vserver security file-directory ntfs sacl show -ntfs-sd sdvol1
vserver security file-directory ntfs sacl show -ntfs-sd sdfolder1
vserver security file-directory ntfs sacl show -ntfs-sd sdfolder2
 
Removing Default DACLs
 
In the above, when we created our NTFS Security Descriptors, these come with a default set of DACLs. We don’t want these default DACLs so firstly we need to remove them. The default DACLs are seen when you run ‘vserver security file-directory ntfs dacl show -ntfs-sd SDNAME’:
 
Account Name           Access Access
--------------         ------ -------
BUILTIN\Administrators allow  full-control
BUILTIN\Users          allow  full-control
CREATOR OWNER          allow  full-control
NT AUTHORITY\SYSTEM    allow  full-control
 
To remove these default DACLs we run the below:
 
vserver security file-directory ntfs dacl remove -vserver svm1 -ntfs-sd sdvol1 -account BUILTIN\Administrators -access-type allow
vserver security file-directory ntfs dacl remove -vserver svm1 -ntfs-sd sdvol1 -account BUILTIN\Users -access-type allow
vserver security file-directory ntfs dacl remove -vserver svm1 -ntfs-sd sdvol1 -account "CREATOR OWNER" -access-type allow
vserver security file-directory ntfs dacl remove -vserver svm1 -ntfs-sd sdvol1 -account "NT AUTHORITY\SYSTEM" -access-type allow
 
vserver security file-directory ntfs dacl remove -vserver svm1 -ntfs-sd sdfolder1 -account BUILTIN\Administrators -access-type allow
vserver security file-directory ntfs dacl remove -vserver svm1 -ntfs-sd sdfolder1 -account BUILTIN\Users -access-type allow
vserver security file-directory ntfs dacl remove -vserver svm1 -ntfs-sd sdfolder1 -account "CREATOR OWNER" -access-type allow
vserver security file-directory ntfs dacl remove -vserver svm1 -ntfs-sd sdfolder1 -account "NT AUTHORITY\SYSTEM" -access-type allow
 
vserver security file-directory ntfs dacl remove -vserver svm1 -ntfs-sd sdfolder2 -account BUILTIN\Administrators -access-type allow
vserver security file-directory ntfs dacl remove -vserver svm1 -ntfs-sd sdfolder2 -account BUILTIN\Users -access-type allow
vserver security file-directory ntfs dacl remove -vserver svm1 -ntfs-sd sdfolder2 -account "CREATOR OWNER" -access-type allow
vserver security file-directory ntfs dacl remove -vserver svm1 -ntfs-sd sdfolder2 -account "NT AUTHORITY\SYSTEM" -access-type allow

And to confirm the DACLs are now clean and empty, run:
 
vserver security file-directory ntfs dacl show -ntfs-sd sdvol1
vserver security file-directory ntfs dacl show -ntfs-sd sdfolder1
vserver security file-directory ntfs dacl show -ntfs-sd sdfolder2
 
Now we are ready to add our DACLs!
 
Creating DACLs
 
To create our DACLs as per ‘Setting up the test environment’:

vserver security file-directory ntfs dacl add -ntfs-sd sdvol1 -vserver svm1 -access-type allow -account Everyone -rights read-and-execute -apply-to this-folder,files
 
vserver security file-directory ntfs dacl add -ntfs-sd sdfolder1 -vserver svm1 -access-type allow -account "DEMO\Domain Admins" -rights full-control -apply-to this-folder,files
vserver security file-directory ntfs dacl add -ntfs-sd sdfolder1 -vserver svm1 -access-type allow -account "DEMO\Domain Users" -rights read-and-execute -apply-to this-folder,files
 
vserver security file-directory ntfs dacl add -ntfs-sd sdfolder2 -vserver svm1 -access-type allow -account "DEMO\Domain Admins" -rights full-control -apply-to this-folder,files
vserver security file-directory ntfs dacl add -ntfs-sd sdfolder2 -vserver svm1 -access-type allow -account "DEMO\Domain Users" -rights modify -apply-to this-folder,files
 
And to confirm the DACLs are as we want, run:
 
vserver security file-directory ntfs dacl show -ntfs-sd sdvol1
vserver security file-directory ntfs dacl show -ntfs-sd sdfolder1
vserver security file-directory ntfs dacl show -ntfs-sd sdfolder2
 
Applying the DACL and SACL
 
We need to create a new ‘vserver security file-directory policy’, add tasks to it (one per level of the volume and folder hierarchy), then finally apply the policy and see it it’s worked! Here the tasks work downwards with the the volume having task index 1, and folders tasks index 2 and 3.
 
vserver security file-directory policy create -policy-name DACL_with_SACL_1 -vserver svm1
vserver security file-directory policy task add -vserver svm1 -path /vol1 -ntfs-sd sdvol1 -policy-name DACL_with_SACL_1 -ntfs-mode replace
vserver security file-directory policy task add -vserver svm1 -path /vol1/folder1 -ntfs-sd sdfolder1 -policy-name DACL_with_SACL_1 -ntfs-mode replace
vserver security file-directory policy task add -vserver svm1 -path /vol1/folder1/folder2 -ntfs-sd sdfolder2 -policy-name DACL_with_SACL_1 -ntfs-mode replace
vserver security file-directory policy task show -vserver svm1 -policy-name DACL_with_SACL_1
vserver security file-directory apply -vserver svm1 -policy-name DACL_with_SACL_1
job show -id 234
 
NOTE: We use ‘ntfs-mode replace’ which replaces permissions. I found ‘ignore’ simply won’t do anything when you run the apply (it does indeed ignore existing ACLs, but also doesn’t apply any of your new ACLs.) Didn’t want to use ‘propragate’ in this scenario (which is the default if you don’t specify ntfs-mode).
 
Did it Work As Expected? NO
 
Unfortunately, it did not work as expected. Yes, the audit SACLs are correct (actually perfect). The problem is that the ‘vserver security file-directory policy apply’ goes and enables inheritance on folders, which is not what we wanted. Which means that my folder2file.txt gets Everyone ‘Read & execute’ access inherited from /vol1. See the picture below.
 
Image: SACLs and DACLs applied using ‘vserver security file-directory’ (click to enlarge)

 
Conclusion
 
‘vserver security file-directory’ is useful for resetting permissions, say when permissions have been lost (like I did with PowerShell in 2015 www.cosonok.com/2015/10/using-data-ontap-apis-powershell-to-set.html), otherwise it is completely the wrong tool for setting NTFS SACLs and modifying DACLs.

...

There is Another Option...

You could configure SLAG (Storage-Level Access Guard). Hopefully I'll get time to do a blog on this in the future, something like 'How to use SLAG for CIFS Auditing on ONTAP'.

“SACLs designate if an object, or sub object, should be audited for a given event, successful or failure. If you are wanting to audit the objects within a volume or qtree you will have 2 options. The easiest would be using SLAG. If the SACLs you are setting are the same across the volume you can use SLAG to apply the SACLs immediately. If you have variance across your qtree or volume when it comes to what you want to audit then you will have to apply the appropriate SACLs to each object you want to audit.”

Monday, 18 January 2021

NetApp VSC, VASA Provider, and SRA virtual appliance for ONTAP - Control Panel

Note: Using NetApp Virtual Storage Console for VMware version 7.2.1P1 here. But I have since checked and this is exactly the same in VSC 9.6.
 
The vSphere Plugin Registration is available at [Capital R!]:
https://{IP or FQDN of your VSC}:8143/Register.html
 
There is also the ‘Control Panel’ available at (typical login username = administrator):
https:// {IP or FQDN of your VSC}:9083
 
Image: NetApp VSC, VASA Provider, and SRA virtual appliance for ONTAP - Control Panel:


The operations available from the Main Menu:

  • Web based CLI interface: Web based access to the command line interface for administrative tasks
  • Support: Generate a file bundle to submit to support
  • Inventory: Listing of all objects and information currently known in SRA Server database
  • Statistics: Listing of all counters and information regarding internal state
  • Right Now: See what operations are in flight right now
  • Logs: Realtime log file access
  • Logout: Logout
Build Release ...
Build Timestamp ...
System up since ...
Current time ...

Unified VSC Web Based CLI Interface Available Commands
  • cluster add
  • cluster delete
  • cluster ensure_pe
  • cluster list
  • cluster listcapabilities
  • cluster listcompliance
  • cluster listpes
  • cluster listprofiles
  • cluster rediscover
  • cluster sfmod
  • vp triggercontaineralarms
  • vp dr_readvvolmetadata
  • vp dr_writemetadata
  • vp dr_readcontainerscpmetadata
  • vp dr_deletecontainerscpmetadata
  • vserver add
  • vserver list
  • vserver delete
  • vp dr_recoverdb
  • vp dr_dbdump
  • container add_storage
  • container create
  • container delete
  • container delete_storage
  • container edit
  • container list
  • container listprofile
  • container liststorage
  • container setdefaultprofile
  • container createbyprofile
  • container resizebyprofile
  • container rebalance
  • profile applytomatchingstorage
  • profile create
  • profile delete
  • profile list
  • profile listcompliance
  • profile liststorage
  • profile listpotentialstorage
  • profile reverseengineer
  • profile set
  • sra processxml
  • vp reloadconfig
  • vp updateconfig
  • vp updategosluntypeconfig
  • vp listconfig
  • vp gosluntypeshow
  • vp annotatelog
  • vcenter register
  • vcenter sync
  • vcenter reloadvms
  • vcenter unregister
  • vcenter gethosts
  • vvol list
  • vvol listnonappdmmanaged
  • vvol listbind
  • vvol listcompliance
  • vvol listinformation
  • vvol liststorage
  • vvol proposeflexvol
  • vvol listattributesize
+ API Commands for simulating VMware interaction:
  • api bindingchangecomplete
  • api bindvirtualvolume
  • api cancelbindingchange
  • api canceltask
  • api clonevirtualvolume
  • api createmetavirtualvolume
  • api createvirtualvolume
  • api createswapvirtualvolume
  • api deletevirtualvolume
  • api fastclonevirtualvolume
  • api getcurrenttask
  • api gettaskupdate
  • api prepareforbindingchange
  • api preparetosnapshotvirtualvolume
  • api querystoragecontainer
  • api queryvirtualvolumeinfo
  • api resizevirtualvolume
  • api revertvirtualvolume
  • api setpecontext
  • api setStorageContainerContext
  • api snapshotvirtualvolume
  • api spacestatsforstoragecontainer
  • api unbindallvirtualvolumefromhost
  • api unbindvirtualvolume
  • api unbindvirtualvolumefromallhost
  • api updatestorageprofileforvirtualvolume
  • api updatevirtualvolumemetadata

From:
"The account used to register VSC to vCenter Server (using https://appliance_ip:8143/Register.html) must be a vCenter Server administrator (assigned to the vCenter Server administrator or administrator role)."