Brief Notes

2017.09.18: ONTAP Special Boot Menu

(1) Normal Boot.
(2) Boot without /etc/rc.
(3) Change password.
(4) Clean configuration and initialize all disks.
(5) Maintenance mode boot.
(6) Update flash from backup config.
(7) Install new software first.
(8) Reboot node.

Selection (1-8)? 22/7

(22/7)     Print this secret List
(23/7)     Boot in nothing initialized mode.
(25/6)     Force boot with multiple filesystem disks missing.
(25/7)     Boot w/ disk labels forced to clean.
(28/7)     Display disk labels.
(28/8)     Examine disk labels in detail.
(29/7)     Bypass media errors.
(30/7)     OBSOLETE: REMOVED. (Was: edit disk labels.)
(6a)       Update flash from alternate backup config.
(4c)       Clean configuration and initialize all disks.
(4d)       Initialize all disks.
(44/4a)    Zero disks if needed and create new flexible root volume.
(44/7)     Assign all disks, Initialize all disks as SPARE, write DDR labels
(readonly) Readonly boot
(prev_cp)  Boot from previous CP
(quotaoff) Boot and turn quotas off

2017.07.10: How to trim spaces from lines in a file

$File = "FILENAME"; $New = @(); get-content $File | foreach{ $New += $_.Trim(" ") }; $New | Out-File $File

2017.04.29: NetAppDocs for 7-Mode via ASUP
FILERA     = Name of Controller A
SERIALA    = Serial Number of Controller A
FILERB     = Name of Controller B
SERIALB    = Serial Number of Controller B
C:\Output\ = Output Folder
HAPAIR     = Output filename (for docx and xlsx)

Get-NtapFilerAsupData -System 'FILERA:SERIAL_NUMBER!FILERB:SERIAL_NUMBER' | Format-NtapFilerData | Out-NtapDocument -WordFile 'C:\Output\HAPAIR.docx' -ExcelFile 'C:\Output\HAPAIR.xlsx'

2017.02.11: PowerShell resolve hostname to IP

2017.01.18: What do storage system and shelf temperature thresholds mean?
system node run -node {NODE-NAME} -command environment chassis list-sensors

2016.12.18: Reset wfa "admin" password
Prompt browse to> {INSTALL_PATH}\WFA\bin
wfa --admin-password
Note: Also useful> wfa -help

2016.12.13: Offline mgmt in Cluster Ring Show
Commands from systemshell% to:
1) Check whether mgwd is running:
spmctl --list | grep mgwd
2) Kill mgwd:
spmctl -h mgwd -s
3) Re-start mgwd:
spmctl -h mgwd -e

2016.09.25: Data collection from Cisco Switches
copy running-config startup-config
show tech-support | no-more
show ssh key
show ssh server
show callhome destination-profile

2016.09.25: Data collection from NetApp Switches (via enable)
write mem
show tech-support
show ip ssh
show logging email config

2016.09.25: Data collection from ONTAP 8.2.3 ::>
rows 0; system health status show; system health subsystem show; system health alert show -instance; cluster peer show -instance; vserver peer show; storage disk show -fields owner,dr-home,array,errors; cluster peer show -instance; cluster peer health show -instance; autosupport invoke -node * -type all -message BASIC_CONFIG

2016.09.25: Data collection from ONTAP 8.3+ ::>
rows 0; system health status show; system health subsystem show; system health alert show -instance; cluster peer show -instance; vserver peer show; storage disk show -fields owner,dr-home,array,errors; cluster peer show -instance; cluster peer health show -instance; autosupport invoke -node * -type all -message BASIC_CONFIG

NetAppdocs instructions:

Import-Module NetappDocs
Find-NtapAsupClusterId -ClusterName CLUSTERNAME -NodeSerialNumber SERIALNUMBER_of_a_NODE
Get-NtapClusterAsupData -Cluster 'CLUSTERNAME:CLUSTER_IDENTIFIER' | Format-NtapClusterData | Out-NtapDocument -WordFile 'C:\Output\CLUSTERNAME.Docx' -ExcelFile 'C:\Output\CLUSTERNAME.xlsx'

PS> One Liner to Parse the Output of> cdpd show-neighbors -v

$cdpd = get-content '.\cdpd output.txt'

$N = $LP = $RD = $RP = ""; "CDOT NODE,LOCAL PORT,REMOTE DEVICE,REMOTE PORT"; $cdpd | foreach{ If($_.StartsWith("Node")){ $N = $_.Substring(5).Trim(" ","`t") }; If($_.StartsWith("Local Port")){ $LP = $_.Substring(11).Trim(" ","`t") }; If($_.StartsWith("Remote Device")){ $RD = $_.Substring(14).Trim(" ","`t") }; If($_.StartsWith("Remote Port")){ $RP = $_.Substring(12).Trim(" ","`t") }; If($RD -and $LP -and $RP){ "$N,$LP,$RD,$RP"; $LP = ""; $RD = ""; $RP = "" } }

Core Clustered Data ONTAP Best Practices TRs

TR-4067: Clustered Data ONTAP NFS Best Practice and Implementation Guide

How to get all 32-bit Snapshots on a 7-Mode  (>= 8.1.4P4 ) System using PowerShell

Import-Module DataONTAP

$Cred = New-Object System.Management.Automation.PsCredential($(Read-Host "U"),$(Read-Host "P" -AsSecureString))

Add-NaCredential -Credential $Cred


Get-NaVol | foreach { $_.Name ; $Snapshots = Get-NaSnapshot -TargetName $_ | Where{ $_.FsBlockFormat -eq "32_bit" }; $Snapshots | FT Name,Created,FsBlockFormat }

Getting all IGROUPS with Mapped LUNs on a Node

import-module D:\NA_DaveC\DataONTAP

$Controller = "CONTROLLER_NAME"
connect-nccontroller $Controller
$NodeName = "NODE_NAME"
(get-nclunmap | where{$_.Node -eq $NodeName}).initiatorgroup | select -uniq

One Liner to Get Contents

$N = @(); (Get-Content "contents.txt") | Foreach{ $N += $_.split(".")[0].Trim(" ") }; $N

How to Trace with Netsh

netsh trace start scenario=NetConnection capture=yes report=yes persistent=no maxsize=1024 correlation=yes traceFile=C:\Logs\NetTrace.etl

netsh trace stop

FUNCTION Split on SemiColon

Function SPLITSEMICOLON{ Param([parameter(Mandatory=$true)][System.Array]$InArr); $Output = @(); $InArr | Foreach { $Output += $_.Split(";"); $Output += ""}; $Output | Set-Content "TEMP.TXT"; Notepad TEMP.TXT }

Boot Loader Boot Options

boot_ontap maint
boot_ontap prompt

Configuring Failover of Node Mgmt in 8.3.X (20151207)
broadcast-domain create -broadcast-domain VLAN123 -mtu 1500
vlan create -node CLU1N1 -vlan-name e1a-123
net int modify -vserver CLU1 -lif CLU1N1_mgmt1 -home-node CLU1N1 -home-port e1a
net int revert -vserver CLU1 -lif CLU1N1_mgmt1
broadcast-domain remove-ports -broadcast-domain Default -ports CLU1N1:e0m
broadcast-domain add-ports -broadcast-domain VLAN123 -ports CLU1N1:e0m,CLU1N1:e1a-123
net int modify -vserver CLU1 -lif CLU1N1_mgmt1 -home-node CLU1N1 -home-port e0m

Power on VM via ESXi CLI (20151207)
vim-cmd vmsvc/getallvms | grep {VM_NAME}
vim-cmd vmsvc/power.getstate {VMID}
vim-cmd vmsvc/power.on {VMID}

Nearstore effect on SnapMirror Priority (2015/11/28)

If Nearstore is enabled on the Controller, the SnapMirror priority will be the same as Client Workloads.
With Nearstore disabled SnapMirror is given a lower priority over Client workload.

Insight 2015 Content Link (2015/11/28)

CMPG (2015/11/28)

cmdownload.exe -filer CONTROLLERNAME -weeks 2
cmpg -analysis -details -f CONTROLLERNAME/*.gz
cmpg -analysis -details -noDataAverage -f CONTROLLERNAME/*.gz

Determining Vol Affinity

In Nodeshell:
FAS> priv set diag
FAS*> vol_db lookup name
The output should be similar to the: f:X:0[0x................]
FAS*> vol_db get waffi_id f:X:0

One Liner PS Function to Convert List to CSV (2015/11/15)
FUNCTION ListToCSV { PARAM([Parameter(Mandatory=$True)][System.Array]$InArr); [String]$OutStr = ""; $InArr | Foreach{ $OutStr += $_.trim() + "," }; $OutStr.Substring(0,$OutStr.Length-1) | Out-File "TEMP.TXT"; Notepad TEMP.TXT }

Copy and paste into PowerShell. Then run -

- and copy and paste in the list. Output appears in Notepad as CSV.

How to collect a SECD trace (2015/11/09)

User to SID Translation (2015/11/09)

$objUser = New-Object System.Security.Principal.NTAccount("DOMAIN\USERNAME")
$strSID = $objUser.Translate

SID to User Translation (2015/11/09)

$objSID = New-Object System.Security.Principal.SecurityIdentifier("S-1-5-21-1234567890-1234567890-1234567890-1234567")
$objUser = $objSID.Translate ( [System.Security.Principal.NTAccount])

20151025: How to get a Count of Unnecessary Default Style Name Maps (cDOT)

PS> (get-ncnamemapping -direction unix_win | where { ($_.replacement.Split("\")[0] -eq "DOMAIN") -and ($_.replacement.Split("\")[2] -eq $_.pattern) }).count

PS> (get-ncnamemapping -direction win_unix | where { ($_.pattern.Split("\")[0] -eq "DOMAIN") -and ($_.pattern.Split("\")[2] -eq $_.replacement) }).count

20151024: Pocket guide for NetApp commands (7-mode)

20151012: Capturing a Perfstat (for Clustered Data ONTAP)

perfstat8 cluster_management_IP_address -m c -t 4 -i 5 -z

20151012: How to capture traces on Data ONTAP 7-Mode systems

pktt start -all -d /etc/crash -b 2m -s 1000m

pktt stop all

20150804: Common Read Only and No Access Export Policies (8.2+)

vserver export-policy create -vserver SVMNAME -policyname expol_noaccess

vserver export-policy create -vserver SVMNAME -policyname expol_readonly
vserver export-policy rule create -vserver SVMNAME -policyname expol_readonly -ruleindex 1 -protocol any -clientmatch -rorule any -rwrule never -anon 65534 -superuser none -allow-suid true -allow-dev true

20150701: Advanced Clustershell - Deleting 7-Mode Snapshots older than 2 Weeks

set -confirmations off
snapshot delete {-is-7-mode true -create-time <2w}
set -confirmations on

20150522: Commands for re-cabling Cluster Network and Increasing from 2 to 4 CIs per FAS8040

R/W Commands Required:

net port modify -node NODENAME -port e0b -role cluster -mtu 9000 -flowcontrol-admin none
net port modify -node NODENAME -port e0d -role cluster -mtu 9000 -flowcontrol-admin none
net int create -vserver NODENAME -lif clus3 -role cluster -home-node NODENAME -home-port e0b -auto true
net int create -vserver NODENAME -lif clus4 -role cluster -home-node NODENAME -home-port e0d -auto true

R/O (Show) Useful Commands on cDOT:

cluster show
set advanced; cluster ring show
set advanced; cluster ping-cluster -node ANYNODE
net port show -role cluster
net int show -role cluster

R/O (Show/User Level) Useful Commands on CN1610 (if replacing ISL cables SFP to Twinax and vice-verse):

show port-channel brief

20150306: How to Install Qual Devices File without CIFS

software get
software get
priv set diag
mv /etc/software/qual_devices_v3 /etc/qual_devices_v3
mv /etc/software/qual_devices_v3.copy /etc/qual_devices_v3.copy
rdfile /etc/qual_devices_v3
priv set admin

20141219: Troubleshooting Cause of Failover via the SP
If not the required information in the AutoSupports, run these from the service processor:
SP> priv set advanced
SP*> system log
SP*> events all
SP*> sp status –d
SP*> system sensors
SP*> version

20141016: A Super Easy One-Line way to get your PSCredential

$Cred = New-Object System.Management.Automation.PsCredential($(Read-Host "U"),$(Read-Host "P" -AsSecureString))

20140421: 7-Mode Flash Cache Checks

# Check if the card is detected by the version of ONTAP, and - if so - whether it is enabled:

sysconfig -ca

# Check flex_scale is licensed:


# Typical options (if it is detected and enabled):

options flexscale.enable             on  
options flexscale.lopri_blocks       off 
options flexscale.normal_data_blocks on  
options flexscale.pcs_high_res       off 
options flexscale.pcs_size           0GB 

# To see it working:

stats show -p flexscale-access -i 1

20140322: Enabling SSH Access to a Clustered ONTAP system Before Completing Cluster Setup

In the below, first break the "Cluster Setup Wizard" by typing "exit" or using Ctrl+C, then run the following commands. The following commands have us create an IP Address for node-mgmt on the node-mgmt port e0M.

network interface create -lif mgmt1 -role node-mgmt -home-node localhost -home-port e0M -address -netmask

security login password -username admin

security login create -username admin -application ssh -authmethod password -role admin

Optional step: For if you need to be able to route to the node-mgmt IP created above before completing the "Cluster Setup Wizard".

system node rename -newname NODE01

network routing-groups route create -vserver NODE01 -routing-group n10.10.10.10/16 -destination -gateway -metric 10

20140117: Checks for non-default SFO (storage failover) settings in Clustered ONTAP
sto fail show -enabled false
sto fail show -auto-giveback true
sto fail show -abort-operations true
sto fail show -check-partner false
sto fail show -onpanic false
sto fail show -onreboot false
sto fail show -auto-giveback-after-panic false

20130427: Disabling 7DOT Timeouts in Console, SSH, Telnet + PuTTY
options autologout.console.enable off
options autologout.telnet.enable off
options ssh.idle.timeout 0
+ Putty > Settings > Connection > SSH > Kex > Max minutes before rekey = 0

20130415: Disabling automatic aggregate Snapshot copy creation

aggr options aggr_name nosnap on
snap delete -A -a aggr_name
snap reserve -A aggr_name 0

20130415: Realtime Monitoring of Data ONTAP with sysstat
From the DOT CLI, run:
sysstat -x 1
Specifically we're interested in the CP ty field. Any b or B's is not good and suggest the storage is being hammered!
CP = Consistency Point
B = Back to back CPs (CP generated CP)
b = Deferred back to back CPs (CP generated CP)

20130415: ESXi 5.* Error: “System logs on host … are stored on non-persistent storage.”
vSphere Client -> Host -> Configuration tab -> Software Panel -> Advanced Settings
Syslog -> global:
Input a value in the form:
Note 1: You do need the square brackets.
Note 2: The logs folder does not need to be created first

20130118: Understanding AD DS Functional & Domain Levels

20121126: Anti-Virus Exceptions for Exchange 2010
On all Exchange servers:
> %ExchangeInstallPath%\
> %systemroot%\system32\inetsrv
On CAS/HT with Witness:
On Mailbox Server - exclude mailbox and logs partition:
> L:\

20120424: Ports needed for Veeam Replication
443 Default VMware web service port
902 VMware data mover port

20120320: ESX/ESXi 4.x Reloading VM Configuration (for some vm-power-on-fails issue)
# vmware-vim-cmd vmsvc/getallvms
# vmware-vim-cmd vmsvc/reload VMID

*on ESXi substitute vim-cmd for vmware-vim-cmd
*also have /power.getstate VMID , / VMID , /power.shutdown VMID , /power.on VMID

20120319: Commands to Monitor Snapshot Deletion
In ESXi 4.1
either # watch "ls -luth *.vmdk"
or if too many snapshots # while true;do date;ls -lht *vmdk|head -10;echo ______;sleep 3;done

20120315: Error when click hyperlink in Outlook "Operation has been cancelled due to restrictions..."
Includes check if String (Default) value of HKEY_CLASSES_ROOT \.html is "htmlfile"

20120313: Windows 7 Event 1511 User Profile Service (caused when local profile is manually deleted)
1: Registry Editor: HKEY_LOCAL_MACHINE > SOFTWARE > Microsoft > Windows NT > CurrentVersion > ProfileList
2: Find the folder name starting with S-1-5 that corresponds to the user account profile that has the error.
3: The folder with .bak needs to be renamed removing the .bak from the end.
4: Set RefCount value = 0
5: Set State value = 0
6: Close Registry Editor, restart computer, and login!

20120226: Enable Bitlocker with USB on Windows 7 system without TPM
gpedit.msc > Computer Configuration > Administrative Templates > Windows Components > BitLocker Drive Encryption > Operating System Drives > Require Additional Authentication at Startup > Enable.

20120124: XenServer 6 – some openvswitch commands (used when looking at ovs-vswitchd high CPU in top)
echo "NR_DOMAIN0_VCPUS=1" > /etc/sysconfig/unplug-vcpus
/etc/init.d/openvswitch restart

20111229: Windows 7 Service Pack Cleanup
dism /online /cleanup-image /spsuperseded

20111221: Test for HTTP / HTTPS connectivity to vSphere vCenter / Host
*Tested works for vSphere 4 & 5
And login with vCenter or Host (root) credentials to display the mob page
Note: http redirects to https if the proxy.xml file is set for httpsWithRedirect. Test works with http and https.

20111209: How to Reboot or Reset a HA Protected VM (HVM) on a Citrix XenServer Host
1 - "xe vm-list" to get the uuid of the VM that is hung
2 - "list_domains" to list the domain uuid's so you can determine the domain # of the VM above by matching the uuids from this output with the uuid for your VM from the previous command.
3 - "/opt/xensource/debug/destroy_domain -domid XX" where XX is the domain number from the previous command
(note: if you don’t run 2 and 3 in a HA environment, you get the error "The operation could not be performed because a domain still exists for the specified VM")
4 - "xe vm-reboot uuid=XXXX --force" where XXXX is the uuid from the first vm-list command for your VM (or “xe vm-reset-powerstate=XXXX --force")

20111208: Enabling Remote SSH on a vSphere 5 Host
Host -> Configuration -> Security Profiles -> Services -> Properties -> SSH -> Options -> Start
Host -> Configuration -> Security Profile -> Firewall -> Properties -> Secure Shell: SSH Server -> Options -> Start

20111201: Determining which memory slots are populated inside a VMware ESXi host using command line
*Applies to ESXi 4.X and 5.X
~ # > mem.out
~ # less -i mem.out
*Scroll down making a note of which DIMMs are listed as being populated

Windows Server 2008
Installing Remote Server and Administration Tools (Vista / Windows 7)
i: Download and install Remote Server and Administration Tools from Microsoft
ii: Control Panel -> Programs and Features -> Turn Windows features on or off -> Tick Remote Server Administration Tools and required options

Exchange 2007

Setting Identities for use with Outlook Anywhere (on CAS server):
Get-ClientAccesServer | FL
Set-ClientAccessServer -Identity "mbx1" –AutodiscoverServiceInternalURI https://nlb.nwtraders.msft/autodiscover/autodiscover.xml
Get-WebServicesVirtualDirectory | FL
Set-WebServicesVirtualDirectory -Identity "mbx1\EWS (Default Web Site)" –InternalUrl https://nlb.nwtraders.msft/EWS/Exchange.asmx
Get-OABVirtualDirectory | FL
Set-OABVirtualDirectory -Identity “mbx1\OAB (Default Web Site)” -InternalURL https://nlb.nwtraders.msft/OAB
Enable-OutlookAnywhere -Server mbx1 -ExternalHostname “nlb.nwtraders.msft” -ClientAuthenticationMethod “NTLM”
Get-ActiveSyncVirtualDirectory | FL
Set-ActiveSyncVirtualDirectory -Identity “mbx1\Microsoft-Server-ActiveSync (Default Web Site)” -InternalURL https://nlb.nwtraders.msft/Microsoft-Server-Activesync
*Can substitute ExternalURL for InternalURL in the above

ESX 3.5
1) Enable root SSH login on an ESX host:
Either create a user and 'Grant shell access' to the user via the VI Client, login to SSH using that account, and “su –” to switch to root user.
or access CLI via console connection using iLO/ DRAC/ KVM/ ….
nano /etc/ssh/shhd_config
Edit the line that starts with PermitRootLogin and change the no to yes.
Press Ctrl-O and then Enter to save the file. Exit with Ctrl-X.
service sshd restart
2) Restart Management Agents on ESX host:
service mgmt-vmware restart
3) Some Deletables:
vmware-XX.log (vmware.log is the active log)
.vmss (these files are generated when the VM is suspended, so if the VM is not currently in a suspended state then safe to delete)
...-zdump.XXX (dump files generated when a guest hangs)
vmware-XX.log.vzsnp (vRanger copies of all files under 1MB that get left behind if a backup job does not complete.)


For disabling platform check for cDOT destination, add following line to /etc/conf/transition-tool.conf and then restart the windows service: cft.skip.clustermode.platform.check = true. Using this flag to disable platform check must not be used without engineering approval (PVR)!