Monday, 18 January 2021

NetApp VSC, VASA Provider, and SRA virtual appliance for ONTAP - Control Panel

Note: Using NetApp Virtual Storage Console for VMware version 7.2.1P1 here. But I have since checked and this is exactly the same in VSC 9.6.
 
The vSphere Plugin Registration is available at:
https://{IP or FQDN of your VSC}:8143/Register.html
 
There is also the ‘Control Panel’ available at (typical login username = administrator):
https:// {IP or FQDN of your VSC}:9083
 
Image: NetApp VSC, VASA Provider, and SRA virtual appliance for ONTAP - Control Panel:


The operations available from the Main Menu:

  • Web based CLI interface: Web based access to the command line interface for administrative tasks
  • Support: Generate a file bundle to submit to support
  • Inventory: Listing of all objects and information currently known in SRA Server database
  • Statistics: Listing of all counters and information regarding internal state
  • Right Now: See what operations are in flight right now
  • Logs: Realtime log file access
  • Logout: Logout
Build Release ...
Build Timestamp ...
System up since ...
Current time ...

Unified VSC Web Based CLI Interface Available Commands
  • cluster add
  • cluster delete
  • cluster ensure_pe
  • cluster list
  • cluster listcapabilities
  • cluster listcompliance
  • cluster listpes
  • cluster listprofiles
  • cluster rediscover
  • cluster sfmod
  • vp triggercontaineralarms
  • vp dr_readvvolmetadata
  • vp dr_writemetadata
  • vp dr_readcontainerscpmetadata
  • vp dr_deletecontainerscpmetadata
  • vserver add
  • vserver list
  • vserver delete
  • vp dr_recoverdb
  • vp dr_dbdump
  • container add_storage
  • container create
  • container delete
  • container delete_storage
  • container edit
  • container list
  • container listprofile
  • container liststorage
  • container setdefaultprofile
  • container createbyprofile
  • container resizebyprofile
  • container rebalance
  • profile applytomatchingstorage
  • profile create
  • profile delete
  • profile list
  • profile listcompliance
  • profile liststorage
  • profile listpotentialstorage
  • profile reverseengineer
  • profile set
  • sra processxml
  • vp reloadconfig
  • vp updateconfig
  • vp updategosluntypeconfig
  • vp listconfig
  • vp gosluntypeshow
  • vp annotatelog
  • vcenter register
  • vcenter sync
  • vcenter reloadvms
  • vcenter unregister
  • vcenter gethosts
  • vvol list
  • vvol listnonappdmmanaged
  • vvol listbind
  • vvol listcompliance
  • vvol listinformation
  • vvol liststorage
  • vvol proposeflexvol
  • vvol listattributesize
+ API Commands for simulating VMware interaction:
  • api bindingchangecomplete
  • api bindvirtualvolume
  • api cancelbindingchange
  • api canceltask
  • api clonevirtualvolume
  • api createmetavirtualvolume
  • api createvirtualvolume
  • api createswapvirtualvolume
  • api deletevirtualvolume
  • api fastclonevirtualvolume
  • api getcurrenttask
  • api gettaskupdate
  • api prepareforbindingchange
  • api preparetosnapshotvirtualvolume
  • api querystoragecontainer
  • api queryvirtualvolumeinfo
  • api resizevirtualvolume
  • api revertvirtualvolume
  • api setpecontext
  • api setStorageContainerContext
  • api snapshotvirtualvolume
  • api spacestatsforstoragecontainer
  • api unbindallvirtualvolumefromhost
  • api unbindvirtualvolume
  • api unbindvirtualvolumefromallhost
  • api updatestorageprofileforvirtualvolume
  • api updatevirtualvolumemetadata

Thursday, 14 January 2021

Unified VSC Appliance: Maintenance Shell/Console Menus, Registration & More...

In this blog post we display the NetApp Unified VSC Appliance (or VMware Tools for ONTAP - Virtual Storage Console, VASA Provider, and SRA) Maintenance Shell/Console Menus and vSphere Plugin Registration web page.

Note: I’m using Version 7.2.1P1 here as that’s what I had in the lab - I doubt these views have changed much in more recent versions.

Unified VSC Appliance: Maintenance Shell/Console Menus

Note: Login username ‘maint’

Main Menu:
----------
1 ) Application Configuration
2 ) System Configuration
3 ) Network Configuration
4 ) Support and Diagnostics

Application Configuration Menu:
-------------------------------
1 ) Display server status summary
2 ) Start Virtual Storage Console service
3 ) Stop Virtual Storage Console service
4 ) Start VASA Provider and SRA service
5 ) Stop VASA Provider and SRA service
6 ) Change 'administrator' user password
7 ) Re-generate certificates
8 ) Hard reset keyStore and certificates
9 ) Hard reset database
10) Change LOG level for Virtual Storage Console service
11) Change LOG level for VASA Provider and SRA service
12) Display TLS configuration
13) Enable TLS protocol
14) Disable TLS protocol

System Configuration Menu:
--------------------------
1 ) Reboot virtual machine
2 ) Shut down virtual machine (Disabled. Must be run on virtual machine console.)
3 ) Change 'maint' user password
4 ) Change time zone
5 ) Change NTP server
6 ) Disable SSH access (Disabled. Must be run on virtual machine console.)
7 ) Increase jail disk size (/jail)
8 ) Upgrade (Disabled. Must be run on virtual machine console.)
9 ) Install VMware Tools (Disabled. Must be run on virtual machine console.)

Network Configuration Menu:
---------------------------
1 ) Display IP address settings
2 ) Change IP address settings
3 ) Display domain name search settings
4 ) Change domain name search settings
5 ) Display static routes
6 ) Change static routes
7 ) Commit changes
8 ) Ping a host
9 ) Restore default settings

Support and Diagnostics Menu:
-----------------------------
1 ) Generate support bundle
2 ) Access diagnostic shell (Disabled. Must be run on virtual machine console.)
3 ) Enable remote diagnostic access (Disabled. Must be run on virtual machine console.)

Note: I’ve not included the:
x ) Exit
b ) Back

vSphere Plugin Registration
 
The vSphere Plugin Registration is available at:
https://{IP or FQDN of your VSC}:8143/Register.html
 
And this is what you’ll see:
 
Image: vSphere Plugin Registration: Plugin service information

 
Image: vSphere Plugin Registration: vCenter Server information

 
Image: vSphere Plugin Registration: vSphere Plugin Unregistration


Register the VASA Provider Extension

Note: This is vSphere 6.7 and VSC 7.2.1P1. The VASA Provider (VP) is enabled by default for new installs since VSC 9.6.

Image: Virtual Storage Console > Configuration > Manage Extensions


Image: Manage Extensions: Enable VASA Provider

You must log out of the vCenter Server, and then log in again to view the configured extensions.

Additional Information

How to tune memory settings of virtual appliance for VSC, VASA Provider, and SRA for scale and performance:
https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/Virtual_Storage_Console_for_VMware_vSphere/tune_memory_settings_of_VM_VSC%2C_VASA_Provider%2C_and_SRA_for_scale_and_performance


Wednesday, 13 January 2021

VMware Virtual Volumes (VVOLs) and NetApp ONTAP

Below are some links to get your started with VVOLs and NetApp ONTAP.
TR-4400 is probably the best starter resource, just it is a little old.
 
Usually, when looking for useful collateral I’ll start with these 3 sites:
1) NetApp Field Portal (NetApp Partners & Employees): https://fieldportal.netapp.com/
2) NetApp Insight 2020 content: https://insightdigital.netapp.com/content-library or Panopto link (for NetApp Partners & Employees)
3) Lab on Demand (mostly NetApp Partners & Employess): https://labondemand.netapp.com/ (also, since there is a big VMware element here, I'd check out https://labs.hol.vmware.com/)
 
Image: VM - Data Path - Protocol Endpoint - VVOL - Storage Container (enabled by VASA Provider and vCenter)


Image: A Couple of Protocol Endpoint LUNs Created by the NetApp VSC Provisioning a VVOL Datastore for SAN consisting of 2 flexvols


Image: Example of deployed files for a VM with one 16GB disk deployed with a VVOL on SAN VM Storage Policy (click to enlarge)

VVOLs on ONTAP: Starter Resources
 
Note: As I write this, 9.7.1P1 is the latest version of the Virtual Storage Console for VMware vSphere.
 
Image: ONTAP VASA Provider architecture

 
TR-4400: VMware vSphere Virtual Volumes with ONTAP
https://www.netapp.com/pdf.html?item=/media/13555-tr4400pdf.pdf

[YouTube] Provisioning a vVols Datastore with the Unified VSC
https://www.youtube.com/watch?v=KNtcVot5yd4

LoD: Virtual Storage Console 9.6 for VMware vSphere 6.7 v1.2

https://labs.hol.vmware.com/HOL-2105-02-HCI - Virtual Volumes (vVols) and Storage Policy Based Management (Not ONTAP but still useful)

Insight 2020: BRK-1262-2 VVOLs Disaster Recovery with ONTAP Tools and VMware SRM 8.3
 
TR-4597: VMware vSphere with ONTAP
https://www.netapp.com/pdf.html?item=/media/13550-tr4597pdf.pdf
 
Insight 2020: BRK-1260-2 Best Practices for VMware vSphere and NetApp ONTAP

LoD: Virtual Storage Console for VMware vSphere 6.7 v1.1-(VSC&VVOL)

LoD: Early Adopter Lab for SnapCenter with VMware vCenters in Enhanced Linked Mode v1.0


VVOLs on ONTAP: Further Resources
 
Docs & Knowledgebase > GPS (Guided Problem Solving) > Management Software > Virtual Storage Console for VMware
http://mysupport.netapp.com/NOW/products/vsc
https://mysupport.netapp.com/GPS/category/ECMLS2588119.html
 
Virtual Storage Console for VMware vSphere: Download:
https://mysupport.netapp.com/site/products/all/details/vsc/downloads-tab
 
Virtual Storage Console for VMware vSphere: Resources:
https://www.netapp.com/support-and-training/documentation/virtual-storage-console-documentation/
 
Virtual Storage Console for VMware vSphere 9.7: Documentation:
https://docs.netapp.com/vapp-97/index.jsp
Configuring vVols datastores:
https://docs.netapp.com/vapp-97/topic/com.netapp.doc.vsc-iag/GUID-74549A9B-2CC0-48AC-885C-4CCC27B7D271.html

Virtual Storage Console for VMware vSphere: Knowledge Base:
https://mysupport.netapp.com/site/article?lang=en&type=guide&page=%2FAdvice_and_Troubleshooting%2FData_Storage_Software%2FVirtual_Storage_Console_for_VMware_vSphere

VVOLs: Other References (Not ONTAP Specific)

Cormac Hogan: VVols (Virtual Volumes) posts:
https://cormachogan.com/virtual-volume-vvols/

Virtual Volumes and Storage Policy-Based Management for Databases:
https://blogs.vmware.com/virtualblocks/2015/12/14/virtual-volumes-and-storage-policy-based-management-for-databases/


VVOLs on ONTAP: Notes

Note: These notes are mostly from TR-4400 which last time I checked was a little old (says July 2018).

  • The VASA Provider requires the ONTAP FlexClone license.
  • The appliance has a built-in watchdog to ensure availability, and it can optionally be configured with the VMware High Availability or Fault Tolerance feature.
  • If the VASA Provider is not available, VMs using VVOLs will continue to run, however, new VVOL datastores and VVOLs cannot be created (VMs using VVOLs cannot be powered on)
  • The appliance should not be on VVOL storage.
  • For dashboard information ... the VASA Provider requires a dedicated installation of OnCommand API Services, and it cannot be shared with multiple VASA Provider instances.
  • Section 2.2 of TR-4400 lists ‘Best Practices
    1. Use the VASA provider for ONTAP to provision VVOL datastores and protocol endpoints.
    2. Evaluate the optimal number of FlexVol volumes for your environment.
    3. Keep Storage Capability Profiles simple.
    4. Consider using Max IOPS to control unknown or test VMs.
  • Using ONTAP VVOLs with vSphere is easy and follows published vSphere methods (see “Working with Virtual Volumes” under “vSphere Storage” in the VMware documentation for the ESXi Server).
  • VMware-managed snapshots are offloaded to space-efficient and fast ONTAP file or LUN clones.
  • One LIF per node for each switch/fabric connection.
  • Add the ONTAP cluster to the Virtual Storage Console under Storage Systems.
  • VVOLs can be protected using Commvault or Veeam. Using: VMware vSphere Storage APIs – Data Protection (formerly known as VMware vStorage APIs for Data Protection or VADP)
  • [2018-05] Two major benefits in using VVOLs with ONTAP (not as compelling with NFS, more so with SAN):
    • Policy-based management to speed VM provisioning and avoid mistakes
    • VM granular management on SAN (performance, QoS, etc)
  • It is a good practice to include multiple FlexVols in a VVOL datastore. Because FlexVols have LUN count restrictions that limit the number of virtual machines, having multiple FlexVols can increase performance (source)
  • Using standard hypervisor-based cloning on a standard datastore, the hypervisor and vCenter take on the overhead for the cloning operation. When cloning on a VVOL backed by NetApp storage, the entire clone creation operation is offloaded to the storage.
  • When a VM is on a VVOL backed by NetApp storage, a VM Snapshot operation in vCenter performs a backup using VMware vCenter, but it is different from a traditional VMware-based snapshot. Using VAAI, the task previously performed by the ESX host is now offloaded to the storage.
  • Steps to Provisioning a VVOL VM (source):
    1. Create a storage capability profile (SCP).
    2. Create a vVols datastore and associate the SCP to the datastore.
    3. Verify the datastore configurations.
    4. Create VM storage policies.
    5. Configure a virtual machine with VM storage policies.
  • New for 2020: VVOLs replication - supported beginning with VP 9.7.1, ONTAP 9.5 and SRM 8.3.
  • No support for VVOLs on NFS v4.1.
  • Virtual Storage Console for VMware renamed to ONTAP Tools for VMware vSphere starting with the 9.8 version.
  • As of ONTAP 9.7, VVOLs are not supported with NVMe-oF ONTAP with vSphere!
  • VMware's vSphere External Storage Evolution/Outlook is to VVOLs (VVOL-NVMe for block, VVOL-NFS for NAS)

Saturday, 9 January 2021

NetApp Certified Hybrid Cloud - Implementation Engineer (NS0-402)

When I was looking through the ‘Reference Document’ for the NetApp certifications (current link), the list of references that most interested me, was the list for ‘NetApp Certified Hybrid Cloud - Implementation Engineer (NS0-402)’. This certification does cover a vast range of  different products and technologies.
 
When I went through the reference links, some did not work (were old - a problem with IT certs in general, stuff goes out of date so quickly), some were duplicated, and some were way too broad to focus studies on; so I’ve reformed the references, removing duplicates and links that don’t work, and trying to put them in some order. The references have been separated them into three categories:
 
1) Blogs, White Papers and Other Articles
2) Documentation Sets
3) Broad References
 
1) Blogs, White Papers and Other Articles
 
Dealing with the Unexpected (Aug. 2019 Blog):
https://netapp.io/2019/08/05/dealing-with-the-unexpected/
 
Automated Data Cloning for Cloud-Based Testing of Software Applications (Feb. 2019 Blog):
https://cloud.netapp.com/blog/automated-data-cloning-for-cloud-based-testing
 
Getting Started with NetApp and Ansible: First Playbook Example (Oct. 2018 Blog):
https://netapp.io/2018/10/11/getting-started-with-netapp-and-ansible-first-playbook-example/
 
Consuming WFA Resources from Ansible (Jul. 2018 Blog):
https://netapp.io/2018/07/11/switching-to-ansible-from-wfa/
 
CloudShip Hybrid Cloud Reference Architecture: Migrating the Workflow Engine to the Cloud (April 2019 White Paper):
https://www.netapp.com/pdf.html?item=/media/19899-wp-7303.pdf
 
NetApp Cloud Insights: A New Way to Monitor Your Cloud Infrastructure (Feb. 2019 White Paper):
https://www.netapp.com/pdf.html?item=/media/12208-wp-cloud-insightspdf.pdf
 
The Twelve-Factor App:
https://12factor.net/
 
JSON Syntax:
https://www.w3schools.com/js/js_json_syntax.asp
 
JSON Data Types:
https://www.w3schools.com/js/js_json_datatypes.asp
 
JSON Arrays:
https://www.w3schools.com/js/js_json_arrays.asp
 
REST API Tutorial: HTTP Methods:
https://restfulapi.net/http-methods/
 
WHAT IS SOURCE CODE MANAGEMENT OR VERSION CONTROL?
https://www.linuxnix.com/what-is-source-code-management-or-version-control/
 
Continuous integration vs. continuous delivery vs. continuous deployment
https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment
 
Kubernetes Concepts:
https://kubernetes.io/docs/concepts/
 
Kubernetes Persistent Volumes:
https://kubernetes.io/docs/concepts/storage/persistent-volumes/
 
GitHub Docs: About pull requests:
https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/about-pull-requests
 
GitHub: Prometheus: Default port allocations:
https://github.com/prometheus/prometheus/wiki/Default-port-allocations
 
NetApp Trident on GitHub:
https://github.com/NetApp/trident
 
Trident for Kubernetes:
https://netapp-trident.readthedocs.io/en/stable-v19.01/kubernetes/index.html
 
Kubernetes Design and Architecture Guide: Concepts and Definitions:
https://netapp-trident.readthedocs.io/en/stable-v19.04/dag/kubernetes/concepts_and_definitions.html
 
Kubernetes Storage Concepts: Persistent Volumes:
https://netapp-trident.readthedocs.io/en/stable-v19.04/dag/kubernetes/concepts_and_definitions.html#kubernetes-storage-concepts
 
StatefulSets:
https://netapp-trident.readthedocs.io/en/stable-v19.04/dag/kubernetes/concepts_and_definitions.html?highlight=shared#statefulsets
 
Kubernetes and Trident Objects: Object Overview:
https://netapp-trident.readthedocs.io/en/stable-v19.04/kubernetes/concepts/objects.html
 
Trident for Kubernetes > Concepts > Virtual Storage Pools:
https://netapp-trident.readthedocs.io/en/stable-v19.07/kubernetes/concepts/virtual_storage_pools.html
 
Trident: Backend configuration:
https://netapp-trident.readthedocs.io/en/stable-v19.07/kubernetes/operations/tasks/backends/index.html#backendconfiguration
 
Trident: Kubernetes: ONTAP (AFF/FAS/Select/Cloud): Choosing a driver:
https://netapp-trident.readthedocs.io/en/stable-v19.07/kubernetes/operations/tasks/backends/ontap.html
 
Trident: Kubernetes: Element (HCI/SolidFire):
https://netapp-trident.readthedocs.io/en/stable-v19.07/kubernetes/operations/tasks/backends/element.html
 
Trident installation modes:
https://netapp-trident.readthedocs.io/en/stable-v19.07/dag/kubernetes/deploying_trident.html#trident-installation-modes
 
Trident: Volume Operations: 8.5.1. Modifying Persistent Volumes:
https://netapp-trident.readthedocs.io/en/stable-v19.07/dag/kubernetes/integrating_trident.html#volume-operations
 
NetApp Trident: Managing Volumes: On-Demand Volume Snapshots
https://netapp-trident.readthedocs.io/en/stable-v19.07/kubernetes/operations/tasks/volumes.html
 
Upgrading Trident on Kubernetes?
https://netapp-trident.readthedocs.io/en/stable-v19.07/support/requirements.html
 
Trident: Docker: Volume Cloning
https://netapp-trident.readthedocs.io/en/latest/docker/use/volumes.html#volume-cloning
 
Cloud Insights > Kubernetes Data Collector:
https://docs.netapp.com/us-en/cloudinsights/task_config_telegraf_kubernetes.html
 
ONTAP port usage on a storage system:
https://docs.netapp.com/ontap-9/index.jsp?topic=%2Fcom.netapp.doc.dot-cm-nmg%2FGUID-49D0B88F-42CF-4766-A688-1C77A0AE8BD5.html
 
Cloud Manager > Roles:
https://docs.netapp.com/us-en/occm/reference_user_roles.html
 
Cloud Manager policies for AWS, Azure, and GCP:
https://mysupport.netapp.com/site/info/cloud-manager-policies
 
Cloud Manager > Encrypting volumes with NetApp encryption solutions:
https://docs.netapp.com/us-en/occm/task_encrypting_volumes.html
 
Cloud Manager > WORM storage:
https://docs.netapp.com/us-en/occm/concept_worm.html#activating-worm-storage
 
Cloud Manager > Improving protection against ransomware:
https://docs.netapp.com/us-en/occm/task_protecting_ransomware.html
 
Cloud Manager > Encryption of data at rest:
https://docs.netapp.com/us-en/occm/concept_security.html#encryption-of-data-at-rest
 
Cloud Manager > Encrypting volumes with NetApp encryption solutions:
https://docs.netapp.com/us-en/occm/task_encrypting_volumes.html
 
Cloud Manager > Networking requirements for the Connector: Outbound Internet Access
https://docs.netapp.com/us-en/occm/reference_networking_cloud_manager.html#connection-to-target-networks
 
Cloud Manager > Replicating Data Between Systems (Replicating Data to and From the Cloud):
https://docs.netapp.com/us-en/occm/task_replicating_data.html#data-replication-requirements
 
Microsoft Azure: About Point-to-Site VPN:
https://docs.microsoft.com/en-us/azure/vpn-gateway/point-to-site-about
 
Guidelines for Azure NetApp Files network planning:
https://docs.microsoft.com/en-us/azure/azure-netapp-files/azure-netapp-files-network-topologies
 
Cloud Manager > Networking requirements for Cloud Volumes ONTAP in AWS:
https://docs.netapp.com/us-en/occm/reference_networking_aws.html
 
AWS VPN > Getting Started:
https://docs.aws.amazon.com/vpn/latest/s2svpn/SetUpVPNConnections.html
 
Cloud Sync > How Cloud Sync works:
https://docs.netapp.com/us-en/cloudsync/concept_architecture.html
 
Cloud Manager > Data tiering overview:
https://docs.netapp.com/us-en/occm/concept_data_tiering.html
 
Cloud Manager > Learn about Cloud Tiering (NetApp Service Connector):
https://docs.netapp.com/us-en/occm/concept_cloud_tiering.html
 
Cloud Manager > Managing data tiering from your clusters (Tiering Data from Additional Clusters):
https://docs.netapp.com/us-en/occm/task_managing_tiering.html
 
Cloud Manager > Replicating data between systems: What Replication Policies Do (Types of Replication Policies):
https://docs.netapp.com/us-en/occm/task_replicating_data.html#what-replication-policies-do
 
Cloud Manager > Discovering (and Managing) ONTAP clusters:
https://docs.netapp.com/us-en/occm/task_discovering_ontap.html
 
Authentication with NetApp Cloud Central
https://docs.netapp.com/us-en/occm/api.html#_authentication_with_netapp_cloud_central
 
Cloud Manager > Getting started with Cloud Volumes ONTAP for Azure:
https://docs.netapp.com/us-en/occm/task_getting_started_azure.html
 
Cloud Manager > Modifying Cloud Volumes ONTAP systems:
https://docs.netapp.com/us-en/occm/task_modifying_ontap_cloud.html
 
Cloud Sync > Managing sync relationships (Changing Sync Schedules):
https://docs.netapp.com/us-en/cloudsync/task_managing_relationships.html
 
Cloud Manager > Provisioning storage (Using FlexCache Volumes to Accelerate Data Access):
https://docs.netapp.com/us-en/occm/task_provisioning_storage.html#using-flexcache-volumes-to-accelerate-data-access
 
Quickstart: Set up Azure NetApp Files and create an NFS volume:
https://docs.microsoft.com/en-us/azure/azure-netapp-files/azure-netapp-files-quickstart-set-up-account-create-volumes?tabs=azure-portal
 
What is Cloud Sync?
https://docs.netapp.com/us-en/cloudsync/faq.html
 
Automating NetApp with Ansible:
https://www.ansible.com/integrations/infrastructure/netapp
 
Ansible Documentation: Using Variables:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html
 
Ansible Documentation: Loops:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html
 
ansible.builtin.template – Template a file out to a remote server:
https://docs.ansible.com/ansible/latest/collections/ansible/builtin/template_module.html
 
community.general.na_ontap_gather_facts – NetApp information gatherer:
https://docs.ansible.com/ansible/latest/collections/community/general/na_ontap_gather_facts_module.html#na-ontapgather-facts-module
 
na_ontap_interface – NetApp ONTAP LIF configuration:
https://docs.ansible.com/ansible/latest/collections/netapp/ontap/na_ontap_interface_module.html
 
Enabling SnapMirror on the Element cluster:
http://docs.netapp.com/ontap-9/index.jsp?topic=%2Fcom.netapp.doc.pow-sdbak%2FGUID-A7215166-3A2E-4832-9A16-23C7A94B2958.html
 
FlexCache in ONTAP: ONTAP 9.8
https://www.netapp.com/pdf.html?item=/media/7336-tr4743pdf.pdf
 
FabricPool Best Practices: ONTAP 9.8
https://www.netapp.com/media/17239-tr4598.pdf
 
SnapMirror Synchronous disaster recovery basics
https://docs.netapp.com/ontap-9/index.jsp?topic=%2Fcom.netapp.doc.pow-dap%2FGUID-5080DA69-478E-40ED-87A8-CA506DD00C9D.html&lang=en
 
Elasticity in Cloud Computing: What It Is, and What It Is Not
https://www.usenix.org/system/files/conference/icac13/icac13_herbst.pdf
 
Amazon Lightsail Instance and Block Storage Service Level Agreement
https://aws.amazon.com/lightsail/sla-lightsail-instances-and-block-storage/
 
NetApp HCI for Private Cloud with Red Hat: NVA Design (May 2019 NVA):
https://www.netapp.com/pdf.html?item=/media/7042-nva1133designpdf.pdf
 
Kubernetes and Trident on NetApp HCI: Considerations and Known Issues:
https://kb.netapp.com/Advice_and_Troubleshooting/Cloud_Services/Trident_Kubernetes/Kubernetes_and_Trident_on_NetApp_HCI%3A_Considerations_and_Known_Issues
 
OpenStack: Image Service Overview
https://docs.openstack.org/glance/queens/install/get-started.html
 
2) Documentation Sets
 
Cloud Insights documentation:
https://docs.netapp.com/us-en/cloudinsights/index.html
 
NetApp API Documentation links for:
NetApp Cloud Central, Cloud Volumes Service for AWS, Cloud Volumes ONTAP, Cloud Sync, Virtual Desktop Service:
https://services.cloud.netapp.com/developer-hub
 
YAML:
https://yaml.org/
 
Ansible Documentation:
https://docs.ansible.com/
 
Git Documentation:
https://git-scm.com/docs
 
ONTAP 9.6 REST API online documentation:
https://library.netapp.com/ecmdocs/ECMLP2856304/html/index.html
 
Kubernetes Documentation:
https://kubernetes.io/docs/home/
 
Puppet Product documentation:
https://puppet.com/docs/
 
AWS Documentation:
https://docs.aws.amazon.com/index.html
 
Docker Documentation:
https://docs.docker.com/
 
StorageGRID 11.2: Grid Primer:
https://library.netapp.com/ecm/ecm_download_file/ECMLP2848265
 
StorageGRID 11.2: Administration Guide:
https://library.netapp.com/ecm/ecm_download_file/ECMLP2848253
 
StorageGRID NAS Bridge 2.2: Administration Guide:
https://library.netapp.com/ecm/ecm_download_file/ECMLP2848245
 
NetApp Product Documentation:
https://www.netapp.com/support-and-training/documentation/
 
Microsoft Documentation:
https://docs.microsoft.com/en-us/
 
3) Broad References
 
Azure NetApp Files:
https://cloud.netapp.com/azure-netapp-files
 
Cloud Sync Service:
https://cloud.netapp.com/cloud-sync-service
 
Cloud Backup Service:
https://cloud.netapp.com/cloud-backup
 
Terraform:
https://www.terraform.io/
 
Kubernetes: Production-Grade Container Orchestration:
https://kubernetes.io/
 
NetApp: thePub:
https://netapp.io/
 
The Linux Juggernaut:
https://www.linuxnix.com/
 
NetApp Knowledge Base:
https://kb.netapp.com/
 
Openstack:
https://www.openstack.org/
 
FlexPod Converged Infrastructure:
https://www.netapp.com/data-storage/flexpod/


Tuesday, 5 January 2021

NetApp Core Certs: NCDA (NS0-161) & NCIE-SAN (NS0-519)

 

Every so many years, I need to refresh my NCDA and NCIE-SAN. Really, if you time it right, you should just be refreshing the NCIE-SAN which refreshes the NCDA also (at least it used to work that way). I actually thought I had let both lapse but see from https://www.certmetrics.com/netapp/ that they don’t actually expire until 2021-Mar-05, which gives some time to prepare. Anyway...

If you’re looking to do the NCDA & NCIE-SAN, the key web pages to look at are:

Since I’m a veteran of doing many of these over the years (2012, 2015, 2018), I tend to skip the recommended NetApp University Web Based / Instructor Led courses, and just focus on the ‘View Exam Topics’ and ‘Reference Document (PDF)’ - essentially I’m only looking for new stuff which I’ve not come across. I have been known to turn up to the exam without doing any pre-study, but it’s best to look out for new stuff.

The reference document is an impressive and lengthy piece of work (covers all the NetApp Certification exams), currently the link (from the above links) is -

https://www.netapp.com/pdf.html?item=/media/12176-ncp-reference-document.pdf

- I tend to look through that and the Exam Topics, see if there’s anything I need to swot up on, and use the appropriate references for the task.

And - from the above links, just written more succinctly - the exam topics are:


Exam Topics for NCDA ONTAP (NS0-161)

Storage Platforms: Describe ...
... knowledge of physical storage systems // ... software-defined on-premises or cloud storage systems // ... how to upgrade or scale ONTAP clusters

Core ONTAP: Describe ...
... ONTAP system management // ... high availability concepts // ... how to manage SVMs

Logical Storage: Describe ...
... how to use logical storage features // ... NetApp storage efficiency features // ... NetApp ONTAP Data Fabric solutions.

Networking: Describe how to use ...
... network components // Demonstrate knowledge of how to troubleshoot network components

SAN Solutions and Connectivity: Describe how to use ...
... SAN solutions // Demonstrate knowledge of how to troubleshoot SAN solutions

NAS Solutions: Describe how to use ...
... NAS solutions // Demonstrate knowledge of how to troubleshoot NAS solutions

Data Protection: Describe how to use ...
... ONTAP data protection solutions // ... SnapMirror // Identify MetroCluster concepts

Security: Describe ...
... protocol security // ... security hardening // ... inflight or at rest encryption // Identify SnapLock concepts

Performance: Demonstrate knowledge of how to ...
... administer ONTAP performance // ... troubleshoot storage system performance.


Exam Topics for NCIE-SAN, ONTAP (NS0-519)

ONTAP SAN solution assessment:
Identify ... all prerequisites for the installation of NetApp system, hosts, and FCP/FCoE switches are verified // ... a detailed inventory of SAN components.
Describe the solution design and the hardware provisioned to meet customer’s requirements and expectations.

ONTAP SAN planning:
Identify environmental and physical considerations. And describe these concepts:
storage controller configuration // switch configuration // host configuration // storage space utilization or management

 ONTAP SAN implementation and configuration: Demonstrate knowledge of ...
... clustered storage system node configuration tasks // ... SVM configuration tasks // ... clustered storage system FCP or NVMe configuration tasks // ... clustered storage system iSCSI configuration tasks // ... storage space utilization or management concepts
Describe how to configure ... switches // ... host systems 

ONTAP SAN implementation testing and troubleshooting: Demonstrate knowledge of ...
... testing tasks // ... troubleshooting tasks


Wednesday, 30 December 2020

Goodbye 2020 and Hello 2021!

Wishing You a Healthy and Prosperous 2021!

Or ‘Live Long and Prosper’ as Mr Spock would say!

Or ‘Peace and Long Life’ as Mr Spock would say if you said to him ‘Live Long and Prosper’!


It has been a very strange year 2020...

.
.
.
.
.
.
.

And that was going to be my end of year 2020 post...

One thing I'll share...

My greatest discovery of 2020!

Check out the works of Alan Watts - some places to start:


Blog Stats in 2020

  •  Cosonok’s IT blog: 248k views (Total Pageviews = 3,423,579)
  • My other IT blog*: 26.7k views (Total Pageviews = 148,211)
  • My car blog: 3.22k views
  • My philosophy blog: 1.78k views

*I do not broadcast my other IT blog (not much anyway). It’s turned into more of a note-y type blog, stuff which in the past I might have added to the ‘Brief Notes’ page (which got too big.) My other IT blog has had a good year, over 120 new posts (compared to just 55 for my main blog.)

How Much Money Have I Made from Blogging All Time?

Something I have never shared is how much money I have made from allowing adverts on my blog (which I think you should allow, as that’s how blogger provides their excellent free service). So, if you are curious how much money you could make from a 3 million + pageviews blog, look below.  

  • Sep 26, 2012: £62.31
  • Dec 28, 2012: £73.93
  • Apr 24, 2013: £78.19
  • Oct 22, 2013: £71.61
  • Jan 22, 2014: £63.14
  • May 28, 2014: £84.39
  • Aug 28, 2014: £60.85
  • Nov 26, 2014: £73.81
  • Feb 23, 2015: £64.06
  • May 27, 2015: £65.55
  • Aug 25, 2015: £63.51
  • Dec 23, 2015: £70.44
  • May 25, 2016: £66.67
  • Nov 21, 2016: £65.98
  • Apr 26, 2017: £64.92
  • Oct 25, 2017: £65.17
  • May 22, 2018: £71.2
  • Jun 25, 2019: £63.36
  • Oct 23, 2020: £60.22
  • TOTAL = £1289.31

2014 and 2015 were very good years. The last 3 years have each seen just one payment.

.
.
.

Happy New Year!

Tuesday, 29 December 2020

NetApp Insight 2020: Cloud Insights Content

 

From the previous post Things To Do If You Have Free Time Over Christmas! I shared the links to the NetApp Insight 2020 content (Content Library or Panopto for NetApp Employees and Partners only).

I have spent a little time over the festive break going over the NetApp Cloud Insights content from NetApp Insight 2020. Below are relevant sessions, further resources, and some bonus items.

 

Cloud Insights NetApp Insight 2020 Content

Note: The timings are in the brackets.

  • MEGA-1676-2 More cloud. Less cost [18:06]
  • BRK-1224-2 ONTAP Management with the Power and Scale of the Cloud [31:51]
  • BRK-1289-2 Monitor Your Hybrid Cloud [43:32]
  • DEM-1315-1 End-to-End Monitoring for NetApp and Multicloud [9:29]
  • DEM-1518-2 Hybrid Cloud Management in NetApp Cloud Insights [11:37] (this demo appears in BRK-1289-2)
  • DEM-1350-2 Awesome Visualizations for NetApp Performance [9:56]
  • SPD-1327-3 Monitor Everything! DIY Agents in Cloud Insights [16:45] (collecting metrics with Telegraf)
  • SPD-1319-3 Integrate Data from ServiceNow into Infrastructure Management [18:50]

 

Cloud Insights with Other Products / Notable Mentions

  • Key-1632 Let's Talk Cloud: How is cloud changing the world? [56:49]
  • MEGA-1679-2 Contain your cabin fever with Kubernetes [19:54]
  • BRK-1229-1 Monitor It All: Spot Cloud Analyzer and Cloud Insights [36:03]
  • BRK-1401-1 Practical Approaches to Implementing Automation in a Hybrid Cloud [1:00:55]
  • BRK-1410-2 How To Build a Cloud-Like as-a-Service IT Model on Premises [40:03]
  • BRK-1378-2 Move to the Cloud and Improve Privacy, Security, and Compliance [42:25]
  • BRK-1328-2 Detect and Stop Ransomware Before It's Too Late [29:37]
  • BRK-1320-3 Find and Fix Your Kubernetes Infrastructure Issues, Fast! [39:55]
  • DEM-1534-1 NetApp AFF A250 Virtual Tour and Demo [7:26]
  • DEM-1517-2 Ransomware Detection in NetApp Cloud Insights [9:03]
  • DEM-1368-2 3D Cloud: Graphic Applications in the Cloud [41:09]
  • SPD-1466-1 Accelerate Your Digital Transformation Journey [18:41]
  • SPD-1427-1 NetApp Private Cloud: Solve for Today, Prepare for Tomorrow [20:42]
  • SPD-1165-1 Build a Better Private Cloud: Performance, Flexibility, TCO [15:58]
  • SPD-1387-2 Virtual Desktop Service: New Functionality Overview [15:30]
  • SPD-1299-2 The Dirty Secrets About Virtual Desktops [16:18]
  • SPD-1362-3 Seven Deadly Sins of EUC/VDI in the Cloud [22:23]

More automation related sessions: 1103 / 1295 / 1316 / 1385 / 1411 

Resources from DEM-1350-2: Awesome Visualizations for NetApp Performance

 

Resources from SPD-1327-3: Monitor Everything! DIY Agents in Cloud Insights

Image: Mapping OCI/CI Web UI to REST API

 

  • OCI: Replace ui/#assets with rest/v1/assets
  • CI: Replace web/# with rest/v1


Useful Additional Information


BONUS: Things You Can Do with Cloud Insights (Dashboards from a Cloud Insights Demo)

Monitoring Value
Monitoring - Assure your services

  • Web App KPIs - Full Stack
  • How does my HCI Environment perform?
  • NetApp HCI Lab Admin - How much headroom do I have?
  • Storage Admin - Which nodes are in high demand?
  • Storage Admin - Which pools are in high demand?
Cost Optimization Value
Cost Optimization - Reduction of Waste

  • Where do I have wasted resources?
  • Virtual Reclamation and Remediation - AWS
  • EBS Volume Performance Optimization
  • On-Prem Storage Optimization
  • Am I Getting The Most From My ONTAP Efficiencies?
Troubleshooting Value
Troubleshooting - Quickly Identifying Where Issues Are

  • What Policy Violations Exist in my Environment?
  • Where Are My Shared Application Resources impacted?
  • When is my NetApp Storage Node Headroom Not Meeting Optimal Utilization?

Dashboards (Full List):  Agent Node Monitoring - Acquisition Units / Application Servers Dashboard / AWS Admin - Where can I reclaim waste?  / AWS Storage Consumers  / Cloud Volumes ONTAP - Node Operational/Optimal Points / Cloud Volumes ONTAP - PrePost Efficiencies / Data Services by Organization / Data Services Dashboard / Infrastructure Usage / Monitoring - Web App KPIs - Full Stack / Monitoring - AWS Admin - Which EC2 are in high demand? / Monitoring - Cloud Volumes Ontap - How is Front End (ONTAP) and Back End (EC2) Performance? / Monitoring - data capacity plus snapshot equals total capacity / Monitoring - How does my HCI Environment perform? / Monitoring - Morning Coffee - Cloud Ops / Monitoring - NetApp HCI Lab Admin - How much headroom do I have? / Monitoring - Overall VM Performance / Monitoring - Storage Admin - Which nodes are in high demand? / Monitoring - Storage Admin - Which pools are in high demand? / Optimize - Am I Getting The Most From My ONTAP Efficiencies? / Optimize - EBS Volume Performance / Optimize - On-Prem Storage Optimization / Optimize - Virtual Reclamation and Remediation - AWS / Optimize - Where do I have wasted resources? (Capacity Focus) / Optimize - Where is CVO Storage Efficiency Helping Me? / Storage Admin - Which pools are in high demand? / Storage Operations Morning Coffee (All Vendors) / Tiering - Efficiencies Achieved per Tier / Tiering - Storage Pool Capacities and OverCommit by Storage and Tier / Tiering - Storage Tier Capacity and Performance / Troubleshooting - What Policy Violations Exist in my Environment? / Troubleshooting - When is my NetApp Storage Node Headroom Not Meeting Optimal Utilization? / Troubleshooting - Where Are My Shared Application Resources Impacted? / VMware Admin - Where are opportunities to right size? / VMware Admin - Where can I potentially reclaim waste?


Useful link for NetApp Employees:
https://netapp.sharepoint.com/sites/CloudDataServices/CloudInsightsProduct/SitePages/Home.aspx
Links to:

  • Cloud Insights Field Enablement
  • Cloud Insights Marketing Documents

Wednesday, 2 December 2020

Things To Do If You Have Free Time Over Christmas!

2020 has been a bit of a quiet year on this blog. 2020 has been a very crazy year. I've been lucky to be very busy with work. Just that the work I've been busy with doesn't make good blog content for this blog, or is not easy/possible to blog about. Also my "Tech Roundup's" have fallen by the wayside (they took time to compile and I never got time to go over the links myself.) Because of the 2020 madness, I have nearly 2 weeks off over Christmas (usually I take holiday earlier in the year), and it's a chance to catch up on some of the things I didn't get to do in 2020, whilst at the same time chilling out and having a merry time!

1) NetApp Insight 2020 - Content Library:
https://insightdigital.netapp.com/content-library


Vision Keynotes, Mega Sessions, and more!
300+ sessions and breakouts available in the content library.

2) Catch up with the Tech ONTAP Podcasts:
https://soundcloud.com/techontap_podcast

3) Take some training courses in areas that interest you! From - for example:

https://www.pluralsight.com/
https://www.udemy.com/
https://acloudguru.com/
https://www.netapp.com/support-and-training/netapp-university/
https://labondemand.netapp.com/

If it interests you, try googling "Best Python Game Course" or "Best DevOps Course".

Also - if you have access to them - the NetApp TechMaster content is great too.

Sunday, 25 October 2020

Mini Tutorial: Using Python to Access the OCI REST API (without using OCI_REST utils)

Open your Python app:

 



The following example simply gets the number of storage arrays on the OCI server ociserver.corp.com:
 
import requests
from base64 import b64encode
import json
 
baseurl = "https://ociserver.corp.com/rest/v1"
 
userAndPass = b64encode(b"yourRestApiUser:Password").decode("ascii")
headers = {'Authorization':'Basic %s' % userAndPass}
 
response = requests.request("GET",baseurl + "/assets/storages/count", headers = headers, verify = False)
 
response
response.text
response.json()
NumberOfStorages = response.json()['value']
print("Number of Storages = " + str(NumberOfStorages))
 
A simple function that does the request and returns the json() output.
 
def easyOciGet(path):
  response = requests.request("GET",baseurl + path, headers = headers, verify = False)
  return response.json()
 
easyOciGet('/assets/storages/count')
 
To get rid of the ‘InsecureRequestWarning’.
  
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
 
easyOciGet('/assets/storages/count')
 
Listing storages.
  
storages = easyOciGet('/assets/storages')
 
storages
print(storages)
 
for storage in storages: print(storage)
for storage in storages: print(storage['name'])
for storage in storages: print(storage['name'].ljust(25, ' ') + storage['model'].ljust(25, ' ') + storage['id'])
 
Listing a storage.
 
easyOciGet('/assets/storages/1234')
 
All pools on a single storage.
 
easyOciGet('/assets/storages/1234/storagePools')
 
Storage nodes on a storage.
 
easyOciGet('/assets/storages/1234?expand=storageNodes')
 
Performance on a storage.
 
easyOciGet('/assets/storages/1234?expand=performance')
 
Everything you can expand a storage by.
 
easyOciGet('/assets/storages/1234?expand=_expands')
 
Other examples:

  • easyOciGet('/assets/storages/1234?expand=_expands,storagePools')
  • easyOciGet('/assets/storages/1234?expand=storagePools.volumes')
  • easyOciGet('/assets/storages/1234?expand=storagePools.volumes.performance')
  • easyOciGet('/assets/storages/1234?expand=performance.history')
 
Other things you can do with the NetApp OCI REST API request:
 
Can filter on times using fromTime and toTime (these are time stamps in milliseconds in the UNIX epoch - check out https://www.unixtimestamp.com/ * 1000):
 
?expand=performance.history&fromTime=XX&toTime=YY
 
Expand, query, sort, fields, limit, offset:
  
/rest/v1/assets/volumes/{id}?expand=masks.host,masks.storagePort
/rest/v1/query?objectType=Volume&sort=storage.name&fields=masks.host,masks.storagePort&limit=5&offset=0