Saturday, 22 July 2017

Tech Round Up 22nd July 2017

Stuff collected in the last month. Mostly links (things to read/watch/listen to - if have time). And some random notes.
With headings: Cloud Backup, Microsoft, NetApp, Security, Swagger

Cloud Backup

An article about cloud backups that touches on insider threat:

Microsoft


And here’s a PowerShell script that downloads all the eBooks for you:
https://github.com/boulavogue/Free_Microsoft_eBook_Giveaway/blob/master/Eligman_DownloadAll_PowerShell.ps1
PS There are 800+ MB of PDFs.


NetApp

Get cloud agility for your data on-premises and in the cloud:

Tech ONTAP Podcasts:

Tech ONTAP Podcasts 88,89,91,92,94,96,97:

Some new Technical Reports (TRs) and NetApp Verified Architecture Documents (NVAs):

Q: Are v2 7-Mode and v2 C-Mode licenses the same?
A: “In Data ONTAP 8.2, the only new license key required to build a cluster is a Cluster Base Key... For licensable ONTAP features, the same license keys used for 7-Mode are also used for clustered ONTAP 8.2.”*
*Slightly old information.

Q: How to check if SnapLock is on a 7-Mode System?
Options check:
licensed_feature.snaplock.enable off      
licensed_feature.snaplock_enterprise.enable off
Also check the follow for snaplock options:
aggr status -v
vol options vol_name

New @ thePub:

Indirect I/O Monitoring: Node-Local stats:
workload_detail_volume counter object - gives the counts and average wait time for visits to the cluster interconnect to do indirect I/O.
workload_volume counter object - if this gives non-zero values for volume not on the node, these ops are indirect.
workload_queue_nblade: ... :delay_cluster_interconnect_wait_time

Indirect I/O Monitoring: ClusterShell QoS Statistics:
qos statistics volume latency show - look for any latency on ‘Cluster’.

Security


Image: Beware of ex-admins or admins going rogue!
Swagger

“The World’s Most Popular API Tooling”

Notes on “DevOps - How to Accelerate Your Software Development Cycles and Gain a Competitive Edge with NetApp”

The titular whitepaper is available from NetApp's Field Portal. Since I like to take notes, here are some notes I took down, a handful of screen grabs, some useful links, and some supplemental links and screen grabs, using the original section titles up to the Appendix!

1) Digital transformation drives acceleration

2) Blending development and operations: DevOps

3) DevOps is all about culture, process and technology

4) Why containers rule the new software world

Image: Containers are not Virtual Machines

Benefits of containers...:
- better portability
- increased speed and agility
- higher density
- less infrastructure
- lower cost

...And their limitation:
“Containers have an ephemeral nature due to their stateless design. Creating and discarding them is quick, easy and perfect for DevOps. However, they lack data persistence, which is critical when DevOps shall embrace enterprise applications.”


5) The role of data management in DevOps

Add data persistence to your Docker containers:






6) 10 good reasons for DevOps with NetApp

01) Zero-Touch Storage*
02) Full Automation
03) OpenStack Advantage
04) Container Integration**
05) Data Protection
06) Production-Like QA Testing
07) Cost Savings
08) Developing Anywhere
09) Consistent Performance
10) Data Mobility

**The certified NetApp Docker Volume Plug-in works for ONTAP, SolidFire and E-Series.

7) Customer success with NetApp


Accelerate the software development lifecycle with the NetApp CodeEasy toolkit:
“The CodeEasy Toolkit is a DevOps methodology using NetApp FlexClone and NetApp Snapshot technologies to dramatically save developer checkout and build time and significantly reduce storage usage. It automates steps to create and manage developer FlexClone workspaces and easily fits into most DevOps environments with few to no changes. NetApp also offers FlexClone integration with Perforce Helix, a comprehensive platform for collaboration and version control.”

Image: NetApp CodeEasy toolkit


“NHN Entertainment is breaking down barriers to entry for game developers, who no longer need to own or manage their own IT infrastructure.”


8) NetApp credentials and benefits

+ Early DevOps adopter with an internally create environment: CodeEasy
+ Employs about 5000 software developers
+ Provides seamless integrations with enterprise-class data management
+ Supports open source and APIs for integration
+ Focuses on automation and self-service on premises and in the cloud
+ Recognized as the leader in data management and data protection (Source: IDC)
+ Enables critical capabilities, including data persistence and a validated technology stack for Docker containers, dynamic storage provisioning in Kubernetes and Infrastructure as Code

To learn more, visit www.netapp.com

APPENDIX: Other Resources


Image: Containers vs. VMs



List of Docker Community Editions:
+ Docker CE for MAC
+ Docker CE for Windows
+ Docker CE for CENTOS Distribution
+ Docker CE for Debian
+ Docker CE for Fedora
+ Docker CE for Ubuntu
+ Docker CE for AWS
+ Docker CE for Azure

List of Docker Enterprise Editions:
For Cloud Providers:
+ AWS
+ Azure
For Servers:
+ CentOS
+ Oracle Linux
+ Ubuntu
+ Windows Server 2016
+ RHEL

Monday, 17 July 2017

Automatically Adding Host Credentials to 7MTT via Batch File

It’s a “feature” of 7MTT (every version), that when your 7MTT server reboots, or when you restart the “NetApp 7-Mode Transition Tool” service, you lose all the cached credentials and have to re-input them. This is a bit of a pain if you’ve got lots of hosts in 7MTT. It’s doubly a pain since 7MTT (from 2.0 I think) schedules the 7 to C SnapMirror transfers, so all the transition SnapMirrors grind to a halt until you re-input those credentials.

The good news is that it’s perfectly possible to write a batch file, and re-add the credentials using this batch file. You could even set a scheduled task to run this batch file, to make sure the credentials are loaded. One issue scheduling this with a batch file is that we have to use plain text passwords (if anyone has a cunning solution around this, please let me know.)

Here’s an example batch file to show how it’s done (adding 2 host credentials). Save as say Add_7MTT_Credentials.bat and double-click to run.

echo "START"
echo mypassword | transition credentials add -h host1_IP_FQDN -u admin
echo mypassword | transition credentials add -h host2_IP_FQDN -u admin
echo "END"
pause

Nice and simple!

Note: Version of 7MTT was 3.2.0.

Image: 7MTT Transition Credentials Management Commands
 
Tip 1: As in the above screenshot, to make your DOS command prompt look nicer, check out this Scott Hanselman post.
Tip 2: Making the PowerShell prompt look nicer is simple:
PS> Function Prompt{"YOUR PROMPT DESIGN HERE"}

How to Quickly Disable IE ESC in Windows Server 2016 using PowerShell

I really hate “Internet Explorer Enhanced Security Configuration”. Whilst I understand the arguments for it - and against - I simply loathe it.
Good thing that we can easily disable it using PowerShell.

Disabling IE ESC

All you need to do is (using keyboard shortcuts here):

1) Press the Windows button
2) Type “po” for PowerShell (you might have to type a bit more - with a vanilla install, Windows PowerShell will be the best match)
3) Press Ctrl+Shift+Enter (shortcut to Administrator: Windows PowerShell)
Note: You may need to click ‘yes’ to the User Account Control prompt.
4) Then copy and paste the following code into Powershell:


$AdminsKey = "SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A7-37EF-4b3f-8CFC-4F3A74704073}"
$UsersKey = "SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A8-37EF-4b3f-8CFC-4F3A74704073}"
$BaseKey = [Microsoft.Win32.RegistryKey]::OpenBaseKey("LocalMachine","Default")

$SubKey = $BaseKey.OpenSubkey($AdminsKey,$true)
$SubKey.SetValue("IsInstalled",0,[Microsoft.Win32.RegistryValueKind]::DWORD)
$SubKey = $BaseKey.OpenSubKey($UsersKey,$true)
$SubKey.SetValue("IsInstalled",0,[Microsoft.Win32.RegistryValueKind]::DWORD)
logoff


Note 1: We have to log off and log on for the settings to take effect.
Note 2: Here we disable IE ESC for Admins and Users.

Enabling IE ESC

If you wanted to enable IE ESC, it’s just (0 changed to 1):


$AdminsKey = "SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A7-37EF-4b3f-8CFC-4F3A74704073}"
$UsersKey = "SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A8-37EF-4b3f-8CFC-4F3A74704073}"
$BaseKey = [Microsoft.Win32.RegistryKey]::OpenBaseKey("LocalMachine","Default")

$SubKey = $BaseKey.OpenSubkey($AdminsKey,$true)
$SubKey.SetValue("IsInstalled",1,[Microsoft.Win32.RegistryValueKind]::DWORD)
$SubKey = $BaseKey.OpenSubKey($UsersKey,$true)
$SubKey.SetValue("IsInstalled",1,[Microsoft.Win32.RegistryValueKind]::DWORD)
logoff


Image: “Internet Explorer Enhanced Security Configuration is not enabled” - hurrah!

Credit: This 4sysops.com post led me in the right direction.

NetApp Site Survey Questionnaire / Template / Checklist

I was asked “have you a template that you use for site surveys?”
Now I’ve done the odd Site Survey myself, but I realize I just had it in my *ahem* head precisely what was needed.

It’s never worth reinventing the wheel, so I dug around for a bit without success to see what’s out there. There was a NetApp official “Site Requirements Guide” that’s dated 2014, and is not going to be updated since everyone is - quite rightly - redirected now to http://hwu.netapp.com/  for site requirements.

I remember from a very old (February 2012) study notes post - http://www.cosonok.com/2012/02/netapp-ns0-502-study-notes-part-34-san.html
- that I wrote:


Prepare site for installation:
- Be able to review implementation flowchart with customer and assign task areas.
- Make contact with datacenter/site personnel.
- Validate equipment move path to installation location.
- Verify site infrastructure including: dual power (and proximity of power drops to the install location), floor space, floor loading plan, HVAC (Heating, ventilation, and air conditioning.)
- Validate logistics plan for staging and installation of equipment.
- Verify Ethernet cabling plan and availability of cable supports.
- Verify fiber cabling plan and availability of cable supports.


I Googled “NetApp site survey form” / “NetApp site survey template” and the only decent result - in the first 2 pages of search results - was Mike Brown’s Preparing for a data center site survery.


NetApp Site Survey Questionnaire / Checklist / Template

Since I don’t post spreadsheets/word docs on this blog, here’s my take on this in blog format.

Before Site Visit

+ Have you got a cabinet layout diagram (excel, visio, ...)?
+ Have you got access to the sales order(s) / bill of materials?
+ Does everything look in order on the bill of materials (sufficient cables and of sufficient length, SFPs, IO cards ...)?
+ Check NetApp Hardware Universe for site requirements for the involved hardware (power requirements, hardware dimensions, compatibility ...)
+ If this is adding to an existing installation, verify Interoperability on the NetApp IMT (the IMT also needs to be checked if this is a new FC SAN installation)
+ Who is the site contact?
+ Has site access been arranged?

Equipment to Bring

+ Identification Documents (if required)
+ Safety Equipment (if required: like steel tipped boots...)
+ Documentation
+ Pen and paper/laptop
+ Tape measure
+ Tools (knife - to open boxes, torch ... and whatever else you think might be useful.)

Site Visit

+ Make contact with datacenter/site personnel.
+ If the new hardware is on site, request an inspection (to make sure all parts are present and as expected, and so that the installation engineers know where to find the parts.)
+ If the new hardware in not on site, discover where it will be stored.
+ Validate equipment move path to installation location.
+ Verify the cabinet(s).
-- Cabinets are not damaged.
-- Cabinets are big enough (front posts are forward enough and there is enough length in the cabinet) to take the hardware.
-- There are no obstructions in the cabinets (like cabling, switches, cooling pipes, other hardware units not in the cabinet layout ...)
-- Cabinet feet are down.
+ Verify site infrastructure including:
-- Power (ideally dual power on separate circuits, plug and socket types, proximity of power drops to the install location, PDUs will not obstruct removal of controller trays out the back of the cabinet for non-disruptive maintenance...)
-- Floor space
-- Floor loading plan
-- HVAC (Heating, ventilation, and air conditioning.)
+ Validate logistics plan for staging and installation of equipment.
+ Verify Ethernet cabling plan, availability of cable supports/tidies and cable lengths.
+ Verify fiber cabling plan, availability of cable supports/tidies and cable lengths.

Post Site Visit

+ Document your observations (what’s good, what’s bad, concerns ...)
+ Distribute to the relevant parties.
+ Follow up where there were concerns needing addressing.

Image: Site Survey

Saturday, 15 July 2017

Exploring ONTAP 9.2 OnCommand System Manager GUI

I’m a bit of a geek and love using the Clustershell CLI, or APIs to manage ONTAP; so much so that it’s easy for me to overlook the OCSM Web UI. The Web UI in ONTAP 9.2 is very good, so it definitely warrants taking a closer look.

News

Applications tab: even with an ONTAP 9.2 Simulator - not just AFF like it was before - the Applications tab is now available to all.
Storage Tiers tab: There’s a Storage Tiers tab which lets you configure an ‘External Capacity Tier’ to the cloud (Note: my SIM is not licensed for FabricPool - we only see StorageGrid.)
Configurations > Cluster Settings > Licenses: In ONTAP 9.2, the Cluster Base License is now deprecated. You don't need a Cluster Base License anymore, as shown in the following screenshot.

Image: {DEPRECATED}-Cluster Base License

Screenshots

Note: Please click images to enlarge.

Tab 1) Dashboard

Image: OnCommand System Manager 9.2: Dashboard

Tab 2) Applications

Image: OnCommand System Manager 9.2: Applications

Tab 3) LUNs

Image: OnCommand System Manager 9.2: LUNs

Tab 4) SVMs

Image: OnCommand System Manager 9.2: SVMs

And if you click on ‘Manage

Image: OnCommand System Manager 9.2: SVMs: Manage

Tab 5) Network

Image: OnCommand System Manager 9.2: Network

Tab 6) Storage Tiers

Image: OnCommand System Manager 9.2: Storage Tiers

And clicking  ‘Add/Configure External Capacity Tier

Image: OnCommand System Manager 9.2: Storage Tiers: Add External Capacity Tier

Tab 7) Hardware and Diagnostics (dropdown)

Image: OnCommand System Manager 9.2: Hardware and Diagnostics

Tab 8) Protection (dropdown)

Image: OnCommand System Manager 9.2: Protection

Tab 9) Configurations

Image: OnCommand System Manager 9.2: Configurations

Note: Notice that you can now do 'Cluster Expansion' from the GUI.

Top Right) Buttons and Dropdowns

Guided Problem Solving (button)
This takes you to NetApp’s ‘Welcome to Guided Problem Solving’ page

Technical Support Chat (button)
This lets you chat online with NetApp Technical Support (automatic case generation)

Help (dropdown)
- OnCommand System Manager Help
- Support (This displays the ‘Supportability Dashboard’ - see image below)
- About NetApp OnCommand System Manager

Administration (dropdown)
- Settings
-- Log Level: OFF/ERROR/WARN/INFO/DEBUG
-- Inactivity Timeout (in minutes) {Can set this to 0 if you don’t want the GUI to timeout}

Sign Out (button)

Search dropdown
- You can search on: All/ Volumes/ LUNs/ Qtrees/ Network Interfaces/ SVMs/ Aggregates/ Disks/ Ethernet Ports

Search textbox

+ (Plus) (button/dropdown)
- Quick access to certain (common?) tasks*: Create Volume/ Create Aggregate/ Create Subnet/ Volume Move/ Volume Resize/ Add Licenses
*I wonder if this is customizable

Image: OnCommand System Manager 9.2: Top Right: Buttons and Dropdowns

Image: OnCommand System Manager 9.2: Supportability Dashboard

SEO: Graphical Documentation OnCommand System Manager 9.2 User Interface

Friday, 14 July 2017

NetApp ONTAP 9.2 Simulator Setup: Part 2 of 2

Continuing from the previous post...

Note: Click to enlarge any of the pictures in this post. 

The Guided Cluster Setup in ONTAP 9.2 is really very good. I remember I tried to setup a 9.1 simulator using the guided setup, and that didn’t work, with ONTAP 9.2 it can be used to setup your simulator.

First,
log into:
https://node_mgmt_IP

And click the big ‘Guided Setup’ button.

Image: Welcome to the Guided Cluster Setup

Image: ‘Guided Setup’ button

1) Cluster

Input:
Cluster name
Node name
admin password
Feature Licenses (Optional)

Then click ‘Submit and Continue

Image: Guided Setup to Configure a Cluster: Cluster and Node name

Image: Guided Setup to Configure a Cluster: admin password and licenses

2) Network

Input:
Cluster Management Address
Node Management Address(es)
Service Processor Address(es)
DNS Details
NTP Details

Then click ‘Submit and Continue

Image: Guided Setup to Configure a Cluster: Network

3) Support

Configure:
AutoSupport
Event Notifications
Cluster Configuration Backup (recommended for single-node clusters)

Then click ‘Submit and Continue

Image: Guided Setup to Configure a Cluster: Support

4) Storage

This step will configure you a recommended storage layout or you can skip.

Click ‘Submit and Continue’ or ‘Skip this step

Image: Guided Setup to Configure a Cluster: Storage

Image: Guided Setup to Configure a Cluster: Storage Recommendation

5) SVM

In this section you can create a Storage Virtual Machine, and configure Data Protocols - CIFS, NFS, iSCSI, FC/FCoE - as required (if you have the license.)

The click ‘Submit and Continue’ or ‘Skip this step

Image: Guided Setup to Configure a Cluster: SVM

6) Summary

And we’re finished!

Click ‘Manage your cluster’ or ‘Export Configuration

Image: Guided Setup to Configure a Cluster: Summary

Finally,
just an image to prove log in into a working ONTAP 9.2 simulator!

Image: OnCommand System Manager: NetApp Release 9.2: Tue Jun 20 03:20:03 UTC 2017
Up next: Exploring the ONTAP 9.2 OnCommand System Manager GUI...

NetApp ONTAP 9.2 Simulator Setup: Part 1 of 2

Note: For a more detailed simulator build, check out this post which I wrote for 8.3.2:

ONTAP 9.2 went GA on Jun 29 2017 (see Software Change Log).

The NetApp ONTAP 9.2 Simulator is not yet out (I checked this morning on the 'Simulate ONTAP for ONTAP 8.x and 9.x' download page), so if you want to play with ONTAP 9.2 GA, you’ll need to upgrade the ONTAP 9.1 simulator, and this is simple to do.

Step-by-Step Guide

Note: This walkthrough is done with VMware Workstation 12.5.

1) Download 92_q_image.tgz from here:

2) Serve the 92_q_image.tgz file from a WebServer

Image: ONTAP 9.2 served (user whatever webserver is approved for use at your organization)

3) Download vsim-netapp-DOT9.1-cm.ova from here:

4) Open the OVA in VMware Workstation, and Import

5) Edit the ‘Virtual Machine Settings’ as desired

6) (Optional) Configure Memory down to 4GB
The SIM comes configured with 5GB, and normally you’d not change this, or increase it to make an upgrade possible. Here we do something slightly different. We reduce the Virtual Machine Memory down to 4096MB and then edit the VMX file with this extra line:

pciHole.start = "1024"

Image: Example of edited VMX file

7) Power on the VM and Press Ctrl-C for Boot Menu (when it appears)

Image: Press Ctrl-C for Boot Menu

8) Select option (7) Install new software first, and follow the prompts:


This procedure is not supported for Non-Disruptive Upgrade on an HA pair. The software will be installed to the alternate image, from which the node is not currently running. Do you want to continue? y

In order to download the package, a temporary network interface needs to be configured.

Select the network port you want to use for the download: e0c

The node needs to reboot for this setting to take effect. Reboot now? y


The node reboots, and then:


In order to download the package, a temporary network interface needs to be configured.

Enter the IP address for port e0c: 10.0.0.2
Enter the netmask for port e0c: 255.0.0.0
Enter IP address of default gateway:

What is the URL for the package? http://10.0.0.1/92_q_image.tgz
What is the username on "10.0.0.1", if any?


Then the install will run (be patient). And then:


Do you want to set the newly installed software as the default to be used for subsequent reboots? y

The node must be rebooted to start using the newly installed software. Do you want to reboot now? y


The node will reboot.

9) When ‘Press Ctrl-C for Boot Menu’ appears, press Ctrl-C

All being well you should have seen the node boot to ONTAP 9.2, and will see something like the below:


bootarg.from.version="9.1"
bootarg.to.version="9.2"


Image: bootarg.to.version="9.2"

Note the boot menu option ‘(9) Configure Advanced Drive Partitioning’ which wasn’t there before.

10) In the Boot Menu, select:


(4) Clean configuration and initialize all disks


And follow the prompts to kick off the wipeconfig request:


Zero disks, reset config and install a new file system? y
This will erase all the data on the disks, are you sure? y


Wait for the wipeconfig request to complete.

11) Cluster Setup Wizard

After the wipeconfig the node will boot into the ‘cluster setup wizard’.


Enabling AutoSupport can significantly speed problem determination and resolution...
Type yes to confirm and continue: yes


Enter the node management interface...
...port
...IP address
...netmask
...default gateway

In part 2 we’ll continue the ONTAP 9.2 Simulator Setup in the WebUI:
https://node_mgmt_IP

Image: Welcome to the cluster setup wizard

Image: Node management interface configuration

Image: Use your web browser to complete cluster setup ... or ... the command line