Friday, 24 February 2017

PowerShell Treesize for Windows (Get-Treesize.ps1)

After my “treesize”-esque programs for NetApp Clustered ONTAP and 7-Mode, it occurred to me that it would be fairly trivial to write a ‘PowerShell Treesize for Windows’. Of course, this is perhaps a slightly pointless endeavour when there is already the excellent TreeSize Free from Jam-Software. One thing I should point out, is that even running in ‘Administrator: Windows PowerShell’, you still might get some access denied (which we don’t get with our NetApp Treesizes run as admin/root.)

Here is a usage example>


.\Get-Treesize.ps1 -ScanStartPath C:\Users


You want to run this in Windows PowerShell remembering to right-click the icon to ‘Run as administrator’.

Image: Running PowerShell Treesize for Windows in ‘Administrator: Windows PowerShell’

And the output is a CSV file (default name = TreesizeReport.CSV, specify -OutputFile to change this name) with these columns:

Order
Directory Level
File Type
Name
Path
File Size (bytes)
Dir. Size (bytes)
Creation Time
Last Access Time
Last Write Time

Note: My personal recommendation is - don’t use this, use TreeSize Free from Jam-Software (and no, I don’t have shares.)

The Script


##################
## Get-Treesize ##
##################

Param(
  [Parameter(Mandatory=$True,HelpMessage="Example: C:\Users")]
  [String]$ScanStartPath,
  [String]$OutputFile = "TreesizeReport.CSV",
  [String]$DateFormat = "yyyy.MM.dd"
);""

## VERIFY $ScanStartPath ##
"INFO: $ScanStartPath verifying ..."
$ScanStartPath = $ScanStartPath.Trim("\")
If(!(Get-ChildItem -Path $ScanStartPath)){"ERROR: $ScanStartPath is incorrect!";EXIT}
"INFO: $ScanStartPath verified.`n"

## GLOBAL VARIABLES ##
[System.Array]$Global:CSV = @() # Output CSV
[Int64]$Global:Order = 0 # For output ordering

## ADD TO CSV FUNCTION ##
Function AddInfoToCSV{
  $Global:Order ++
  $Global:CSV += [PSCustomObject]@{
    "Order"             = $Global:Order
    "Directory Level"   = $Level
    "File Type"         = "file"
    "Name"              = $G.Name
    "Path"              = $Path
    "File Size (bytes)" = $G.Length
    "Dir. Size (bytes)" = ""
    "Creation Time"     = $G.CreationTime.ToString($DateFormat)
    "Last Access Time"  = $G.LastAccessTime.ToString($DateFormat)
    "Last Write Time"   = $G.LastWriteTime.ToString($DateFormat)
  }    
  Return $G.Length
}

## RECURSIVE SCANNING FUNCTION ##
Function GetDirInfoRecursive{
  Param([String]$Path)
  Write-Host "." -NoNewLine
 
  ## PATH MANIPULATIONS ##
  [System.Array]$PathSplit = $Path.Split("\")
  [Int]$level = $PathSplit.count
  [String]$CurrentDir = $PathSplit[$level - 1]
  [String]$SubPath = ""
  For($i = 0; $i -lt ($level -1); $i++){$SubPath += $PathSplit[$i] + "\"}         
 
  ## CHECKING CURRENT DIRECTORY ##
  $CurrDirInf = (Get-ChildItem -Path $SubPath | Where{$_.Name -eq $CurrentDir})
  If(!$CurrDirInf){$CurrDirInf = (Get-ChildItem -Path $SubPath -Hidden | Where{$_.Name -eq $CurrentDir})}
  If(!$CurrDirInf){Return 0}
 
  ## GETTING FILES AND DIRECTORIES ##
  $GetFiles       = Get-ChildItem -Path $Path -File
  $GetHiddenFiles = Get-ChildItem -Path $Path -File -Hidden
  $GetDirectories = Get-ChildItem -Path $Path -Directory
  $GetHiddenDirs  = Get-ChildItem -Path $Path -Directory -Hidden
 
  ## ORDER AND INITIALIZE DIRSIZE ##
  $Global:Order ++
  $Local:Order = $Global:Order
  $Local:DirSize = 0
 
  ## CYCLE THROUGH DIRECTORIES AND FILES ##
  Foreach($G in $GetDirectories){$Local:DirSize += (GetDirInfoRecursive ($Path + "\" + $G.Name))}
  Foreach($G in $GetHiddenDirs) {$Local:DirSize += (GetDirInfoRecursive ($Path + "\" + $G.Name))}
  Foreach($G in $GetFiles)      {$Local:DirSize += (AddInfoToCSV $G) }
  Foreach($G in $GetHiddenFiles){$Local:DirSize += (AddInfoToCSV $G) }
 
  ## ADD DIRECTORY INFORMATION TO CSV ##
  $Global:CSV += [PSCustomObject]@{
    "Order"             = $Local:Order
    "Directory Level"   = $Level
    "File Type"         = "directory"
    "Name"              = $CurrentDir
    "Path"              = $Path
    "File Size (bytes)" = ""
    "Dir. Size (bytes)" = $Local:DirSize
    "Creation Time"     = $CurrDirInf.CreationTime.ToString($DateFormat)
    "Last Access Time"  = $CurrDirInf.LastAccessTime.ToString($DateFormat)
    "Last Write Time"   = $CurrDirInf.LastWriteTime.ToString($DateFormat) 
  }
  Return $Local:DirSize
}

##################
## MAIN PROGRAM ##
##################

("START: " + [String](Get-Date))
[Void](GetDirInfoRecursive $ScanStartPath);""
("END: " + [String](Get-Date) + "`n")
$Global:CSV | Sort "Order" | Export-CSV "$OutputFile" -NoTypeInformation


Tuesday, 21 February 2017

A Noob’s First Experience of Amazon Web Services

Initial Setup

This seems like a good place to start:

+ Click on the “Create a Free Account”.
+ Register as a new user (or you can use your Amazon.com shopping account)
+ Personal Account: Fill in the required information.
+ Payment Information (even though this is the AWS Free Tier, still need valid Payment Information)
+ Identity Verification
+ Support Plan: Basic
+ Personalize Your AWS Experience

Simples!

Login to Console

Image: AWS Console
Under ‘Build a solution’:

- Launch a virtual machine (with EC2)
- Build a web app (with Elastic Beanstalk)
- Deploy a serverless microservice (with Lambda, API Gateway)
- Host a static website (with S3, CloudFront, Route 53)
- Create a backend for your mobile app (with Mobile Hub)
- Register a domain (with Route 53)

Under ‘Learn to build’:

**Loads of tutorials, labs, videos and project guides**

- Websites (3 tutorials, 3 labs, 3 videos, 6 project guides)
- DevOps (2 tutorials, 3 labs, 6 videos, 3 project guides)
- Backup & Recovery (3 project guides, 2 tutorials, 3 labs, 3 videos)
- Big Data (2 tutorials, 3 labs, 3 video, 3 project guides)
- Databases (5 tutorials, 3 labs, 3 videos, 2 project guides)
- Mobile (7 getting started guides,  1 lab, 3 videos)
- Internet of Things
- High Performance Computing

Via the Services drop down:

**A massive array of available services (70 services across 17 categories)**

Compute
- EC2
- EC2 Container Service
- Lightsail
- Elastic Beanstalk
- Lambda
- Batch

Storage
- S3
- EFS
- Glacier
- Storage Gateway

Database
- RDS
- DynamoDB
- ElastiCache
- Redshift

Networking & Content Delivery
- VPC
- CloudFront
- Direct Connect
- Route 53

Migration
- DMS
- Server Migration
- Snowball

Developer Tools
- CodeCommit
- CodeBuild
- CodeDeploy
- CodePipeline

Management Tools
- CloudWatch
- CloudFormation
- CloudTrail
- Config
- OpsWorks
- Service Cataolg
- Trusted Advisor
- Managed Services
- Application Discovery Service

Security, Identity & Compliance
- IAM
- Inspector
- Certificate Manager
- Directory Service
- WAF & Shield
- Compliance Reports

Analytics
- Athena
- EMR
- CloudSearch
- Elasticsearch Service
- Kinesis
- Data Pipeline
- QuickSight

Artificial Intelligence
- Lex
- Polly
- Rekognition
- Machine Learning

Internet Of Things
- AWS IoT

Game Development
- GameLift

Mobile Services
- Mobile Hub
- Cognito
- Device Farm
- Mobile Analytics
- Pinpoint

Application Services
- Step Functions
- SWF
- API Gateway
- Elastic Transcoder

Messaging
- SQS
- SNS
- SES

Business Productivity
- WorkDocs
- WorkMail
- Amazon Chime

Desktop & App Streaming
- WorkSpaces
- AppStream 2.0

First Impressions

Wow, powerful stuff!

Some Links



Using Ruby (on Windows) with NetApp ONTAP - Dummy’s Guide

... and - to complete a trilogy - here’s my (Dummy’s) guide for Ruby with ONTAP!

1) Download Ruby

This seems like a good place to start:

I downloaded Ruby 2.3.3 (x64):

2) Install Ruby

Double-click the rubyinstaller exe file, and follow the prompts to install.
The Ruby 2.3 64-bit default install directory is C:\Ruby23-x64.
If it’s your only version of Ruby, tick the ‘Add Ruby executables to your PATH’ - in fact, tick everything, why not!

Image: Installing Ruby for Windows

3) Download the NetApp Manageability SDK (NMSDK)
If not already done so...

> NetApp Manageability SDK > All Platforms

4) NetApp Ruby Library

Unzip the NMSDK.
To keep my Ruby libraries in one place, I copy the NetApp folder from:

netapp-manageability-sdk-5.6 > lib > ruby > NetApp

- to -

C:\Ruby23-x64\lib

5) Test

Below is a very simple Ruby script example taken from the NMSDK and ZExplore.

i) Copy and paste into a text editor
ii) Edit CLUSTER_NAME, USER_NAME, and PASSWORD as required.
iii) Save as get-version.rb
iv) Run in the DOS command prompt as>
ruby get-version.rb


$:.unshift 'C:/Ruby23-x64/lib/NetApp'
require 'NaServer'

s = NaServer.new("CLUSTER_NAME", 1 , 31)
s.set_server_type("FILER")
s.set_transport_type("HTTPS")
s.set_port(443)
s.set_style("LOGIN")
s.set_admin_user("USER_NAME", "PASSWORD")

api = NaElement.new("system-get-version")

xo = s.invoke_elem(api)
if (xo.results_status().eql?("failed"))
  print ("Error:\n")
  print (xo.sprintf())
  exit
end
print ("Received:\n")
print (xo.sprintf())


Image: Ruby success!

THE END

Note: Interestingly, I didn’t get an SSL error with Ruby (but I did with Python.)

Monday, 20 February 2017

Using Python (on Windows) with NetApp ONTAP - Dummy’s Guide

Following on from the previous post, my (Dummy’s) guide for Python with ONTAP!

1) Download Python

This seems like a good place to start: The Hitchhiker's Guide to Python

2) Install Python

Double-click the python-2.7.12.msi and follow the prompts to install.
- Default installation directory is C:\Python27\
- On the Customize Python 2.7.12 screen, I select ‘Add python.exe to Path’ (since I have no other version of Python).

Image: Installing Python for Windows

3) Download the NetApp Manageability SDK (NMSDK)
If not already done so...

> NetApp Manageability SDK > All Platforms

4) NetApp Python Library

Unzip the NMSDK.
To keep my python libraries in one place, I copy the NetApp folder from:

netapp-manageability-sdk-5.6 > lib > python > NetApp

- to -

C:\Python27\Lib

5) Test

Below is a very simple Python script example taken from the NMSDK and ZExplore.

i) Copy and paste into a text editor
ii) Edit CLUSTER_NAME, USER_NAME, and PASSWORD as required.
iii) Save as get-version.py
iv) Run in the DOS command prompt as>
python get-version.py


import sys
sys.path.append("C:/Python27/Lib/NetApp")
from NaServer import *

s = NaServer("CLUSTER_NAME", 1 , 31)
s.set_server_type("FILER")
s.set_transport_type("HTTPS")
s.set_port(443)
s.set_style("LOGIN")
s.set_admin_user("USER_NAME", "PASSWORD")

api = NaElement("system-get-version")

xo = s.invoke_elem(api)
if (xo.results_status() == "failed") :
  print ("Error:\n")
  print (xo.sprintf())
  sys.exit (1)

print ("Received:\n")
print (xo.sprintf())


Note: I was getting the error -
status="failed" reason="[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)" errno="13001"
- and had to change transport_type to HTTP, and port to 80, to get this to work.

Image: Python success!

THE END

Random tip: To change the DOS command prompt to "CMD> ", type in the command prompt>
prompt CMD$g$s

Sunday, 19 February 2017

Using Perl (on Windows) with NetApp ONTAP - Dummy’s Guide

If you’re a regular reader of this blog, you’ll know I love a bit of PowerShell - I’ve tagged 128 posts with PowerShell. Perl - on the other hand - has yet to be tagged on one post. Well, this changes today! The titular “dummy” is me (I’m a Perl noob), and - since I work predominantly from Windows laptop/workstation - this is going to be Perl on Windows...

1) Install Perl

Firstly, we need to install Perl. The following link - http://learn.perl.org/installing/ - contains links to Installing Perl on Windows, OSX, and Unix/Linux.


Image: Padre on Strawberry Perl installed!

Image: Command Prompt/PowerShell and ‘perl -v’ to verify installation.

Image: App-cpanminus installed too.

2) Download the NetApp Manageability SDK (NMSDK)

> NetApp Manageability SDK > All Platforms

3) NetApp Perl Modules

Unzip the NMSDK.
To put all my Perl libraries in one place, I copy the NetApp folder from:

netapp-manageability-sdk-5.6 > lib > perl > NetApp

- to -

C:/Strawberry/perl/lib

4) Test

Below is a very simple Perl script example taken from the NMSDK and ZExplore.

i) Copy and paste into a text editor
ii) Edit CLUSTER_NAME, USER_NAME, and PASSWORD as required.
iii) Save as get-version.pl
iv) Run in the DOS command prompt as>
perl get-version.pl


require 5.6.1;
use lib 'C:/Strawberry/perl/lib/NetApp';
use strict;
use warnings;
use NaServer;
use NaElement;

my $s = new NaServer('CLUSTER_NAME', 1 , 31);
$s->set_server_type('FILER');
$s->set_transport_type('HTTPS');
$s->set_port(443);
$s->set_style('LOGIN');
$s->set_admin_user('USER_NAME', 'PASSWORD');

my $api = new NaElement('system-get-version');

my $xo = $s->invoke_elem($api);
if ($xo->results_status() eq 'failed') {
  print 'Error:\n';
  print $xo->sprintf();
  exit 1;
}

print 'Received:\n';
print $xo->sprintf();


Note: I received an error - errno="13001" - using Cluster Management IP address, so added the name of the cluster to my hosts file and this worked.

Image: Perl success!

THE END