Sunday, 30 October 2016

Simulating Linux SSH in PowerShell for ONTAP

There’s no native way in Windows of doing> ssh [user@]hostname command
So - as a curiosity - I thought I’d write the function. This works for NetApp Clustered Data ONTAP/ONTAP.

The Script

Save as say ssh.ps1 and import into your PowerShell session using . .\ssh.ps1 (dot space dot), then run in PowerShell as>
ssh [user@]hostname command


# The following function simulates the Linux SSH syntax in PowerShell for ...
# ... NetApp Clustered Data ONTAP, in conjunction with the Data ONTAP PSTK:
# > ssh [user@]hostname command
# It takes advantage of the NcCredential feature of the Data ONTAP PSTK ...
# ... use> Add-NcCredential CONTROLLER = Add credentials

Function SSH{
  ## GENERIC: LOAD THE DATA ONTAP PSTK ##
  If(!(Get-Module DataONTAP)){
    [Void](Import-Module DataONTAP -ErrorAction SilentlyContinue)
    If(!(Get-Module DataONTAP)){ "Failed to load DataONTAP PSTK!"; RETURN }
  }
 
  ## SCENARIO 1: No Argument/No 2nd Arg (No $Args[0]/No $Args[1]) ##
  If(!$Args[0]){ "SYNTAX ERROR: 0 arguments detected (2 expected)!"; RETURN }
  If(!$Args[1]){ "SYNTAX ERROR: 1 argument detected (2 expected)!"; RETURN }
 
  ## PROCESS INPUT $Arg[0] and $Arg[1] ##
  [System.Array]$Arg0 = $Args[0].Split("@")
  If($Arg0.Count -eq 2){
    [String]$User = $Arg0[0]
    [String]$Host = $Arg0[1]
  }elseif($Arg0.Count -eq 1){
    [String]$User = ""
    [String]$Host = $Arg0[0]
  }else{
    "SYNTAX ERROR: Too many @ in $Args[0]!"; RETURN
  }
 
  ## CHECK HOST CREDENTIAL ##
  $GetNcCred = Get-NcCredential $Host
  If(!$GetNcCred){
    "ERROR: No credentials for $Host in the NcCredentials cache. To add use> Add-NcCredential $Host"; RETURN
  }
  $NcCredUser = $GetNcCred.Credential.Username
  If($User -and ($NcCredUser -ne $User)){
    "ERROR: NcCredential for $Host uses user $NcCredUser and not $User. To add correct credential use> Add-NcCredential $Host"; RETURN
  }
 
  ## EXAMINE CurrentNcController ##     
  If($Global:CurrentNcController){
    If($Global:CurrentNcController.Name -ne $Host){ $Global:CurrentNcController = $Null}
    else{
      $TempUser = $Global:CurrentNcController.Credentials.Username
      $TempDomain = $Global:CurrentNcController.Credentials.Domain
      If($TempDomain){ $TempUser += ("\" + $TempDomain) }
      If($User -ne $TempUser){ $Global:CurrentNcController = $Null }
    }
  }
 
  ## CONNECT ##
  If(!$Global:CurrentNcController){
    [Void](Connect-NcController $Host -ErrorAction SilentlyContinue)
    If(!$Global:CurrentNcController){"ERROR: Failed to connnect to $Host!"; RETURN}
  }
 
  ## RUN COMMAND ##
  (Invoke-NcSsh $Args[1]).Value
}


Example

Image: Running SSH [user@]hostname command in PowerShell
Note: Not sure if it’s because I’m using a simulator, but this doesn’t run fast, the Invoke-NcSSH isn’t quick.

Wednesday, 26 October 2016

Offbox Anti-Virus Configuration Super Express Guide (8.3.2)

This guide covers a configuration on the NetApp cluster for Offbox Anti-Virus, with a view to a non-multi-tenancy/non-service-provider environment where we’ll configure just one scanner-pool and an on-access-policy on the cluster/admin SVM, and use these for any Data SVM requiring Vscan.

Part 1) Cluster Build

1.1) Create a security login for the Anti-Virus user::>


security login create -username LAB\AVUSER -application ontapi -authmethod domain -role readonly -vserver CLUSTERNAME


1.2) Create a scanner pool::>


vserver vscan scanner-pool create -vserver CLUSTERNAME -scanner-pool POOLNAME -servers VSCAN_SERVER_IPADDRESSES -privileged-users LAB\AVUSER


1.3) Create an on-access-policy (or use the default default_CIFS on-access-policy)::>


vserver vscan on-access-policy create -vserver CLUSTERNAME -policy-name POLICYNAME -filters FILTERS


{Configure your on-access-policy as per requirements}

Table: Vscan on-access-policy settings and defaults
Part 2) SVM Build

2.1) Apply the scanner-pool to the Data SVM::>


vserver vscan scanner-pool apply-policy -vserver DATASVM -scanner-pool POOLNAME -scanner-policy primary


2.2) Disable the default_CIFS on-access-policy (if not using), and enable the desired on-access-policy::>


vserver vscan on-access-policy disable -vserver DATASVM -policy-name default_CIFS
vserver vscan on-access-policy enable -vserver DATASVM -policy-name POLICYNAME


2.3) Enable Vscan on the SVM::>


vserver vscan enable -vserver DATASVM


2.4) Configure shares with the -vscan-fileop-profile to enable scanning::>

::> cifs share modify -vscan-fileop-profile ?
no-scan     = Virus scans are never triggered for accesses to this share.
standard    = Virus scans can be triggered by open, close, and rename operations.
Strict      = Virus scans can be triggered by open, read, close, and rename operations.
writes-only = Virus scans can be triggered only when a file that has been modified is closed.

Part 3) Vscan Infrastructure Build

See the NetApp Interoperability Matrix for infrastructure components and Anti-Virus vendors documentation.

As an example with McAfee:

- A very rough rule of thumb is that you’ll need one AV server for every 6000 CIFS IO/s (please check but 2 CPUs and 8GB RAM is a reasonable server spec)
- Vscan Server’s O/S = Windows Server 2008 or better (not Server 2016 yet)
- McAfee VirusScan Enterprise for Storage 1.2.0
- Clustered Data ONTAP 8.3.2
- Clustered Data ONTAP Antivirus Connector 1.0.3
- McAfee Vscan timeout needs to be set to 25s (want McAfee to timeout before ONTAP)

Tuesday, 25 October 2016

ONTAP 8.3.2 Defaults - Vserver Services NDMP

In the 8th of the ONTAP 8.3.2 defaults series and following on from the previous post about NDMP, we look at Vserver Services NDMP defaults.

All the information presented in the table below can be got from::>


set diag
man vserver services ndmp modify


Image: Table of Vserver Services NDMP defaults

And in CSV format (actually - Hash delimited format because of commas in the table):


Switch#Priv.#Values#Default#Note
 -vserver##{vserver name>##
 -maxversion##{integer>##
 -ignore-ctime-enabled##{true|false>#false#
 -offset-map-enable##{true|false>#true#
 -tcpnodelay ##{true|false>#false#
 -tcpwinsize##{integer>#32768#(32K)
 -data-port-range##{text>#all#[1024-65535]
 -backup-log-enable##{true|false>#true#
 -per-qtree-exclude-enable##{true|false>#false#
 -authtype##{NDMP Authentication types>, ...#challenge#
 -debug-enable#adv.#{true|false>#false#
 -debug-filter#adv.#{text>#none#
 -dump-logical-find#adv.#{text>#default#
 -abort-on-disk-error#adv.#{true|false>#false#
 -fh-dir-retry-interval#adv.#{integer>#250#milliseconds
 -fh-node-retry-interval#adv.#{integer>#250#milliseconds
 -restore-vm-cache-size#adv.#{integer>#64#
 -dump-detailed-stats#diag.#{true|false>#false#
 -enable##{true|false>#false#
 -preferred-interface-role##{cluster|data|node-mgmt|intercluster|cluster-mgmt>, ...#intercluster, cluster-mgmt, node-mgmt#Data Vserver: intercluster, data
 -secondary-debug-filter#adv.#{text>##
 -is-secure-control-connection-enabled##{true|false>#false#


NDMP Configuration Super Express Guide (8.3.2)

Since I’m lazy to keep re-reading PDFs, and my memory’s not so great, here is my “NDMP Configuration Super Express Guide” ...

This is based on the official NDMP Configuration Express Guide (for ONTAP 8.3.2), with a view to using something like Symantec Netbackup 7.7 which supports CAB (Cluster Aware Backup) extension and operates in SVM scope NDMP mode, and not using tape (at least not having tape directly connected to a NetApp controller.)

1) Verify the backup application supports the Cluster Aware Backup (CAB) extension from the NetApp Interoperability Matrix tool.

2) Verify every node in the cluster (which has volumes we want to backup) has intercluster LIFs::>


network interface show -role intercluster


And check they are ping-able from the backup server (also check you can ping the/a Cluster Management LIF).

3) Enable SVM-scoped NDMP mode (disable node-scope mode)::>


system services ndmp node-scope-mode off


4) Enable NDMP service on the admin SVM::>


vserver services ndmp on -vserver {CLUSTER NAME}


And verify the NDMP service is enabled and view the settings::>


vserver services ndmp show -vserver {CLUSTER NAME}


5) Configure a backup user for the cluster::>


security login create -user-or-group-name {BACKUP ADMIN} -application ssh -authmethod {AUTH METHOD} -role backup


6) Generate NDMP password for the user (not the same as the user's login password) and record this password::>


vserver services ndmp generate-password -vserver {CLUSTER NAME} -user {BACKUP ADMIN}


7) Point your backup application at the/a Cluster Management LIF, with the specified backup user, and create backup jobs/policies, and test as required.

THE END

Sunday, 23 October 2016

CollectCC: C-Mode Cluster and Vservers Data Collector (Clustershell Commands)

Like the 7-Mode version of the data collector tool from this post - Collect7: 7-Mode vFiler0 and vFilers Data Collector - here it’s just rewritten for Clustered Data ONTAP (probably not as useful for C-Mode, since there’s no config files to collect).

You point it at your Cluster, and it will collect the outputs of all the specified ClusterShell commands, for the Cluster and any and all Vservers, and store the outputs in a folder for the Cluster, and a folder each for the Vservers. A couple of examples of using it are below:

Example 1: Running as .\CollectCC.ps1

Example 2: Running as .\CollectCC.ps1 -Cluster 10.0.1.100 -UserName admin -Password $PW

The Script

Copy into a text editior and save as CollectCC.ps1. Edit the sections -
CLUSTER CLI OUTPUTS TO COLLECT
VSERVER CLI OUTPUTS TO COLLECT
- as per requirements.


#############################################################################
## COLLECTCC: C-MODE CLUSTER AND VSERVERS DATA COLLECTOR (CSHELL COMMANDS) ##
#############################################################################

Param(
  [Parameter(Mandatory=$True)][String]$Cluster,
  [Parameter(Mandatory=$True)][String]$UserName,
  [Parameter(Mandatory=$True)][SecureString]$Password
) # Tip: You can use> $PW = Read-Host -AsSecureString, as input for -Password

FUNCTION Wr{ Param([String]$P="",[String]$C="WHITE"); Write-Host $P -ForegroundColor $C}; Wr
FUNCTION Get-CmCliOutput{
  Wr "Getting the output of::> $COMMAND " CYAN
  If(!$V){ [String]$StrOut = Invoke-NcSsh -Command $COMMAND }
  else{ [String]$StrOut = Invoke-NcSsh -Command "vserver context $VSERVER; $COMMAND" }
  If(!$StrOut){ RETURN }
  [System.Array]($StrOut.Split("`n")) | Set-Content ($SaveDir + "/CLI_OUT_" + $COMMAND)
}
FUNCTION CollectC{
  Param([System.Array]$COMMANDS,[Switch]$V)
  If(!$V){ [String]$SaveDir = $Cluster }
  Else{ [String]$SaveDir = ($Cluster + "." + $VSERVER) }
  [Void](New-Item -Path $SaveDir -ItemType Directory -Force)
  Foreach($COMMAND in $COMMANDS){ [Void](Get-CmCliOutput) };Wr
}

<# CLUSTER CLI OUTPUTS TO COLLECT: #>; [System.Array]$CCommands = `
"cluster show",`
"node show",`
"vserver show"

<# VSERVER CLI OUTPUTS TO COLLECT: #>; [System.Array]$VCommands = `
"cifs show",`
"nfs show",`
"cifs share show",`
"export-policy rule show"

## MAIN PROGRAM ##

If(!(Get-Module DataONTAP)){ [Void](Import-Module DataONTAP -ErrorAction SilentlyContinue) }
If(!(Get-Module DataONTAP)){ Wr "Failed to load DataONTAP PSTK!" RED; EXIT }
Wr "Loaded DataONTAP PSTK" GREEN; Wr
Wr "<<<<< COLLECTCC: C-MODE CLUSTER AND VSERVERS DATA COLLECTOR (CSHELL COMMANDS) >>>>>" MAGENTA; Wr
$Cred = New-Object System.Management.Automation.PsCredential($UserName,$Password)
[Void](Connect-NcController $Cluster -Credential $Cred -ErrorAction SilentlyContinue)
If(!$Global:CurrentNcController){ Wr "Failed to connnect to $Cluster!" RED; EXIT }
Wr "Connected to $Cluster" GREEN
$query = Get-NcVserver -template
$query.State = "running"
[System.Array]$Vservers = (Get-NcVserver -Query $query).Vserver
Wr "Collecting data for cluster $Cluster" GREEN
[Void](CollectC $CCommands)
Foreach($VSERVER in $Vservers){
  Wr "Collecting data for cluster $Cluster and Vserver $VSERVER" GREEN
  [Void](CollectC $VCommands -V)
}


ONTAP 8.3.2 Defaults – NFS Export-Policy Rule

In the 7th of the ONTAP 8.3.2 defaults series (final one for now…), we look at NFS Export-Policy Rule defaults.

All the information presented in the table below can be got from::>


set diag
man export-policy rule modify


Image: Table of Export-Policy Rule defaults

And in CSV format:


Switch,Priv.,Values,Default,Note
 -vserver,,{vserver name},,
 -policyname,,{export policy name},,
 -ruleindex,,{integer},,
 -protocol,,{Client Access Protocol},any,
 -clientmatch,,{text},,
 -rorule,,{authentication method},,
 -rwrule,,{authentication method},,
 -anon ,,{text},65534,
 -superuser,,{authentication method},none,
 -allow-suid,,true|false,true,
 -allow-dev,,true|false,true,
 -ntfs-unix-security-ops,adv.,ignore|fail,fail,
 -chown-mode,adv.,restricted|unrestricted,restricted,


ONTAP 8.3.2 Defaults – CIFS Share

In the 6th of the ONTAP 8.3.2 defaults series, we look at CIFS Share defaults.

All the information presented in the table below can be got from::>


set diag
man cifs share create
man cifs share modify


Image: Table of CIFS Share defaults

And in CSV format:


Switch,Priv.,Values,Default,Note
 -vserver,,{vserver name},,
 -share-name,,{Share},,
 -path,,{text},,
 -share-properties,,{share properties},oplocks,browsable,changenotify,
 -symlink-properties,,enable|hide|read-only|symlinks|symlinks-and-widelinks|disable,symlinks,
 -file-umask,,{Octal Integer},,
 -dir-umask,,{Octal Integer},,
 -comment,,{text},,
 -attribute-cache-ttl,,{integer}s,,
 -offline-files,,none|manual|documents|programs,manual,
 -vscan-fileop-profile,,no-scan|standard|strict|writes-only,standard,
 -max-connections-per-share,,{integer},4294967295,
 -force-group-for-create,,{text}, "",


ONTAP 8.3.2 Defaults – CIFS Options

In the 5th of the ONTAP 8.3.2 defaults series, we look at CIFS Options defaults.

All the information presented in the table below can be got from::>


set diag
man cifs options modify


Image: Table of CIFS Options defaults

And in CSV format:


Switch,Priv.,Values,Default,Note
 -vserver,,,,
 -default-unix-user,,,pcuser,
 -read-grants-exec,,enabled|disabled,disabled,
 -wins-servers,,,…,,
 -smb2-enabled,adv.,true|false,true,
 -smb3-enabled,adv.,true|false,true,
 -max-mpx,adv.,,255,
 -shadowcopy-dir-depth,adv.,,5,
 -copy-offload-enabled,adv.,true|false,true,
 -is-copy-offload-direct-copy-enabled,adv.,true|false,true,
 -default-unix-group,,,,
 -shadowcopy-enabled,adv.,true|false,true,
 -is-referral-enabled,adv.,true|false,false,
 -is-local-auth-enabled,adv.,true|false,true,
 -is-local-users-and-groups-enabled,adv.,true|false,true,
 -is-use-junctions-as-reparse-points-enabled,adv.,true|false,true,
 -is-exportpolicy-enabled,adv.,true|false,false,
 -is-unix-nt-acl-enabled,adv.,true|false,true,
 -is-trusted-domain-enum-search-enabled,adv.,true|false,true,
 -client-session-timeout,,,900,(seconds)
 -is-dac-enabled,adv.,true|false,false,
 -restrict-anonymous,adv.,no-restriction|no-enumeration|no-access,no-restriction,
 -is-read-only-delete-enabled,,enabled|disabled,disabled,
 -file-system-sector-size,adv.,512|4096 (in bytes),4096,
 -is-fake-open-enabled,adv.,true|false,true,
 -is-unix-extensions-enabled,adv.,true|false,false,
 -is-search-short-names-enabled,adv.,true|false,false,
 -is-advanced-sparse-file-support-enabled,adv.,true|false,true,
 -max-file-write-zero-length,diag.,[KB|MB],32MB,
 -guest-unix-user,adv.,,,
 -smb1-max-buffer-size,adv.,,65535,
 -max-same-user-sessions-per-connection,adv.,,2050,
 -max-same-tree-connect-per-session ,adv.,,4096,
 -max-opens-same-file-per-tree ,adv.,,800,
 -max-watches-set-per-tree,adv.,,100,
 -is-admin-users-mapped-to-root-enabled,adv.,true|false,true,
 -is-advertise-dfs-enabled,adv.,true|false,false,
 -grant-unix-group-perms-to-others,adv.,true|false,false,


ONTAP 8.3.2 Defaults – CIFS Security

In the 4th of the ONTAP 8.3.2 defaults series, we look at CIFS Security defaults.

All the information presented in the table below can be got from::>


set diag
man cifs security modify


Image: Table of CIFS Security defaults

And in CSV format:


Switch,Priv.,Values,Default,Note
 -vserver,,,,
 -kerberos-clock-skew,,,5,(minutes)
 -kerberos-ticket-age,,,10,(hours)
 -kerberos-renew-age,,,7,(days)
 -kerberos-kdc-timeout,,,3,(seconds)
 -is-signing-required,,true|false,false,
 -is-password-complexity-required,,true|false,true,
 -use-start-tls-for-ad-ldap ,,true|false,false,
 -is-aes-encryption-enabled,,true|false,false,
 -lm-compatibility-level,,lm-ntlm-ntlmv2-krb|ntlm-ntlmv2-krb|ntlmv2-krb|krb,lm-ntlm-ntlmv2-krb,
 -is-smb-encryption-required ,,true|false,false,
 -is-bypass-traverse-checking-enabled ,diag.,true|false,true,



Saturday, 22 October 2016

ONTAP 8.3.2 Defaults – NFS Server

In the 3rd of the ONTAP 8.3.2 defaults series, we look at NFS Server defaults. There are a lot of modify-able NFS server settings (especially since I’m including advanced and diag level settings for completeness), this is not complexity though - since in most cases the settings will be mostly perfectly fine set to defaults - it’s configurability/flexibility/enablement/power for if/when you need to set things differently to default.

All the information presented in the table below can be got from::>


set diag
man nfs server modify


Image: Table of NFS Server defaults – Part 1

Image: Table of NFS Server defaults – Part 2

And in CSV format:


Switch,Priv.,Values,Default,Note
 -vserver,,,,
 -access,,true|false,true,
 -rpcsec-ctx-high,adv.,,0,
 -rpcsec-ctx-idle,adv.,,0,
 -v3,,enabled|disabled,enabled,
 -v4,,enabled|disabled,enabled,
 -udp,,enabled|disabled,enabled,
 -tcp,,enabled|disabled,enabled,
 -default-win-user,,, -,
 -enable-ejukebox,adv.,true|false,true,
 -v3-require-read-attributes,adv.,true|false,false,
 -v3-fsid-change,adv.,enabled|disabled,enabled,
 -v3-connection-drop,adv.,enabled|disabled,enabled,
 -ntfs-unix-security-ops,adv.,fail|ignore|use-export-policy,ignore,
 -chown-mode,adv.,restricted|unrestricted|use-export-policy,use_export_policy,
 -force-spinnp-readdir,diag.,true|false,false,
 -trace-enabled,,true|false,false,
 -trigger,adv.,,60,
 -udp-max-xfer-size,adv.,,32768,
 -tcp-max-xfer-size,adv.,,65536,
 -v3-tcp-max-read-size,adv.,,65536,
 -v3-tcp-max-write-size,adv.,,65536,
 -v4.0-acl,,enabled|disabled,disabled,
 -v4.0-read-delegation,,enabled|disabled,disabled,
 -v4.0-write-delegation,,enabled|disabled,disabled,
 -v4-fsid-change,adv.,enabled|disabled,enabled,
 -v4.0-referrals,adv.,enabled|disabled,disabled,
 -v4-id-domain,,,,
 -v4-validate-symlinkdata ,adv.,enabled|disabled,disabled,
 -v4-lease-seconds,adv.,,30,(seconds)
 -v4-grace-seconds,,,45,(seconds)
 -v4-acl-preserve,,enabled|disabled,enabled,
 -v4.1,,enabled|disabled,enabled,
 -rquota,,enabled|disabled,disabled,
 -v4.1-implementation-domain,adv.,,,
 -v4.1-implementation-name,adv.,,,
 -v4.1-implementation-date,adv.,,,
 -v4.1-pnfs,,enabled|disabled,enabled,
 -v4.0-migration,diag.,enabled|disabled,disabled,
 -v4.1-referrals,adv.,enabled|disabled,disabled,
 -v4.1-migration,diag.,enabled|disabled,disabled,
 -v4.1-acl,,enabled|disabled,disabled,
 -vstorage,,enabled|disabled,disabled,
 -v4-numeric-ids,,enabled|disabled,enabled,
 -default-win-group,,, -,
 -v4.1-read-delegation,,enabled|disabled,disabled,
 -v4.1-write-delegation,,enabled|disabled,disabled,
 -v4.x-session-num-slots,adv.,,180,
 -v4.x-session-slot-reply-cache-size,adv.,,640,(bytes)
 -v4-acl-max-aces,adv.,,400,
 -mount-rootonly,,enabled|disabled,enabled,
 -nfs-rootonly,,enabled|disabled,disabled,
 -auth-sys-extended-groups,adv.,enabled|disabled,disabled,
 -extended-groups-limit,adv.,,32,
 -validate-qtree-export,adv.,enabled|disabled,enabled,
 -mountd-port,adv.,,635,
 -nlm-port,adv.,,4045,
 -nsm-port,adv.,,4046,
 -rquotad-port,adv.,,4049,
 -permitted-enc-types,,,…,des,des3,aes-128,aes-256,
 -showmount,,enabled|disabled,disabled,
 -name-service-lookup-protocol,,TCP|UDP,UDP,
 -map-unknown-uid-to-default-windows-user,adv.,enabled|disabled,enabled,
 -netgroup-dns-domain-search,adv.,enabled|disabled,enabled,
 -netgroup-trust-any-ns-switch-no-match,adv.,enabled|disabled,disabled,
 -ntacl-display-permissive-perms,adv.,enabled|disabled,disabled,
 -v3-ms-dos-client,,enabled|disabled,disabled,
 -ignore-nt-acl-for-root,adv.,enabled|disabled,disabled,
 -cached-cred-positive-ttl,adv.,,86400000,
 -cached-cred-negative-ttl,adv.,,7200000,
 -cached-transient-err-ttl,diag.,,30000,
 -skip-root-owner-write-perm-check,adv.,enabled|disabled,disabled,