Sunday, 25 October 2020

Mini Tutorial: Using Python to Access the OCI REST API (without using OCI_REST utils)

Open your Python app:


The following example simply gets the number of storage arrays on the OCI server
import requests
from base64 import b64encode
import json
baseurl = ""
userAndPass = b64encode(b"yourRestApiUser:Password").decode("ascii")
headers = {'Authorization':'Basic %s' % userAndPass}
response = requests.request("GET",baseurl + "/assets/storages/count", headers = headers, verify = False)
NumberOfStorages = response.json()['value']
print("Number of Storages = " + str(NumberOfStorages))
A simple function that does the request and returns the json() output.
def easyOciGet(path):
  response = requests.request("GET",baseurl + path, headers = headers, verify = False)
  return response.json()
To get rid of the ‘InsecureRequestWarning’.
import urllib3
Listing storages.
storages = easyOciGet('/assets/storages')
for storage in storages: print(storage)
for storage in storages: print(storage['name'])
for storage in storages: print(storage['name'].ljust(25, ' ') + storage['model'].ljust(25, ' ') + storage['id'])
Listing a storage.
All pools on a single storage.
Storage nodes on a storage.
Performance on a storage.
Everything you can expand a storage by.
Other examples:

  • easyOciGet('/assets/storages/1234?expand=_expands,storagePools')
  • easyOciGet('/assets/storages/1234?expand=storagePools.volumes')
  • easyOciGet('/assets/storages/1234?expand=storagePools.volumes.performance')
  • easyOciGet('/assets/storages/1234?expand=performance.history')
Other things you can do with the NetApp OCI REST API request:
Can filter on times using fromTime and toTime (these are time stamps in milliseconds in the UNIX epoch - check out * 1000):
Expand, query, sort, fields, limit, offset:

Tuesday, 6 October 2020

[Python] Translating Curl into Python

I needed to Python-ize some code I had written for a Delphix Reporting integration with NetApp’s OCI. This KB for Delphix Reporting uses curl to login to the API and get data:

curl --data "password=YOURPASS&user=YOURUSER" http://YOURSERVER/api/login


Note: is a nice online curl to Python converter (do not enter real data; just use it to get the syntax right.)


Translated into Python, the above should be:


import requests


data = {

  'password': 'YOURPASS',

  'user': 'YOURUSER'



response ='https://YOURSERVER/api/login', data=data)


Unfortunately, I don’t have success when I try to connect to Delphix Reporting.


ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)


I had to put verify=False at the end, like so.


import requests


data = {

  'password': 'YOURPASS',

  'user': 'YOURUSER



response ='https://YOURSERVER/api/login', data=data, verify=False)


Great, we have our response but how do we access our required output (loginToken & userId)?


import json

json_data = response.json()

loginToken = json_data["loginToken"]

userId = json_data["userId"]


Great, so now we know how to get our loginToken and userId. How do we get Storage Summary data?


From the KB:


curl -H "X-Login-Token: LOGINTOKEN" -H "X-User-Id: USERNAME" "http://YOURSERVER/api/get_report?report=result_storage_summary"


This translates to:


headers = {

  'X-login-token': loginToken,

  'X-user-id': userId



params = (




response = requests.get('https://YOURSERVER/api/get_report', headers=headers, params=params, verify=False)


Note: Remember the False in verify=False is case sensitive – it must have a capital F.


And this worked. To get the output into JSON format, run:


json_data = response.json()


And to see the JSON format output, run:



Saturday, 12 September 2020

How to Find a Specific Column Name (e.g Custom Annotation) in the NetApp OCI DWH

The following is quite a nice query if you're trying to find everywhere a particular column name appears in the NetApp OnCommand Insight Data Warehouse MySQL database. Perhaps you are looking for a custom annotation, and want to find the table(s) that annotation presents itself in. The SQL query searches through all column names, in all tables, in all schemas, in the OCI DWH. In the example below we use country as the custom annotation we are searching for (the search is not case sensitive.)


(SELECT * FROM information_schema.columns WHERE table_schema = 'dwh_capacity'

UNION SELECT * FROM information_schema.columns WHERE table_schema = 'dwh_capacity_efficiency'

UNION SELECT * FROM information_schema.columns WHERE table_schema = 'dwh_cloud_cost'

UNION SELECT * FROM information_schema.columns WHERE table_schema = 'dwh_custom'

UNION SELECT * FROM information_schema.columns WHERE table_schema = 'dwh_fs_util'

UNION SELECT * FROM information_schema.columns WHERE table_schema = 'dwh_inventory'

UNION SELECT * FROM information_schema.columns WHERE table_schema = 'dwh_performance'

UNION SELECT * FROM information_schema.columns WHERE table_schema = 'dwh_ports'

UNION SELECT * FROM information_schema.columns WHERE table_schema = 'dwh_reports') as t0

WHERE COLUMN_NAME = 'Country'; -- for example...

Image: Search all Columns in all Tables in all Schemas (MySQL)

NetApp DWH SQL: Find Latest Updated/Created Tables in dwh_custom and test

Along similar lines, something else that is useful if you've inherited a DWH SQL instance with many tables in dwh_custom or test that you're not sure what they're for or if they are even used any more.

select TABLE_SCHEMA,TABLE_NAME,CREATE_TIME,UPDATE_TIME,TABLE_COMMENT from information_schema.tables where TABLE_SCHEMA = 'dwh_custom' order by UPDATE_TIME desc;

select TABLE_SCHEMA,TABLE_NAME,CREATE_TIME,UPDATE_TIME,TABLE_COMMENT from information_schema.tables where TABLE_SCHEMA = 'dwh_custom' order by CREATE_TIME desc;

select TABLE_SCHEMA,TABLE_NAME,CREATE_TIME,UPDATE_TIME,TABLE_COMMENT from information_schema.tables where TABLE_SCHEMA = 'test' order by UPDATE_TIME desc;

select TABLE_SCHEMA,TABLE_NAME,CREATE_TIME,UPDATE_TIME,TABLE_COMMENT from information_schema.tables where TABLE_SCHEMA = 'test' order by CREATE_TIME desc;

Friday, 11 September 2020

Python 3: OCI REST API - Set Annotation On Storage Array

In the NetApp OCI API Samples for Python (available from the OCI Operational Server WebUI > REST API documentation > API samples) there is a which can be used to set the annotation on a specific volume. With a slight modification of the Python, we get a which can be used to set the annotation on a specific storage. An example of using it>

python --url https://YOUR_OCI_SERVER --user YOUR_REST_USER --password YOUR_PASSWORD --storage STORAGE_NAME -annotation SOME_ANNOTATION --value ANNOTATION_VALUE

Image: Insight REST API documentation > API samples

The Script

Save as say ‘’.

#!/usr/bin/env python 


Set annotation for a specific storage


from __future__ import print_function

import json

from oci_rest import OciRest, configure_command_line_parser

def find_storage(oci, storage_name):

  for storage in oci.get('assets/storages'):

    if storage['name'] == storage_name:

      return storage['self']

  return None

def annotate_storage(oci, storage_url, annotation_name, annotation_value):

  return oci.put('{}/annotations'.format(storage_url), data=json.dumps(



        "rawValue": annotation_value,

        "definition": {"name": annotation_name}




if __name__ == "__main__":

  # Get the default command-line arguments (url, user, password)

  parser = configure_command_line_parser(usage='Set storage annotation')


  # Add additional arguments to configure the search criteria

  parser.add_argument('--storage', required=True, help="Storage name")

  parser.add_argument('--annotation', required=True, help="Annotation name (do NOT use label!)")

  parser.add_argument('--value', required=True, help="Annotation value to set")


  options = parser.parse_args()

  url, user, password = options.url, options.user, options.password


  with OciRest(url, user, password) as oci:

    storage_url = find_storage(oci,


    if storage_url is not None:

      new_annotation = annotate_storage(oci,





      print('Created new annotation:\n', json.dumps(new_annotation))



      print("Could not find storage {}".format(

APPENDIX: List of the 14 out-of-the-box ‘NetApp OCI API Samples for Python’

  • - print list of all storagePools with storage name and total allocatedCapacity
  • - manipulate annotation types
  • - manipulate business entities
  • - Print status report for each datasource
  • - Create OCI user
  • - print list of storage pools filtered by storage tier, and thresholds
  • - print virtual machines with dataStores on specific storage
  • - Programmatically manipulate OCI license
  • - Import datasource patch
  • - Set annotation for a specific volume
  • - Set datacenter annotation for hosts based on their connected storage annotation
  • - Set annotation for a group of volumes identified with prefix
  • - set values for LDAP configuration
  • - create, delete agents and integrations. Ingest integration data.

Sunday, 6 September 2020

Python 3: Basic Website Monitor

A bit like the multiple ping test monitor I wrote in PowerShell back in 2015. Here’s my Python 3: Basic Website Monitor.

Very simple to use, just save the script below as say and then run the script like -


- where the arguments are the website you want to monitor (no limits on the number of website you want to monitor.)

In the example below I’ve run -


(with some intentional spelling mistakes)

Image: Example of the Python 3 Basic Website Monitor in action

If you missing some of the Python libraries, check out the previous post here

The Script


This program expects websites as arguments, and will add https:// to the name.

Example> python


import sys

import time

import colorama

import urllib.request

from termcolor import *


def webCheck(url):

  url = "https://" + url


    status_code = urllib.request.urlopen(url).getcode()

    return("OK(" + str(status_code) + ")")

  except urllib.error.URLError:


# Create status array...

status = [''] * len(sys.argv)

x = 1

for webSite in sys.argv[1::]:

  status[x] = "UNTESTED"

  x += 1

chkNext = 1 # Website to check next

loop = "Y"

while loop:

  print(chr(27) + "[2J") # CLS

  x = 1

  for webSite in sys.argv[1::]:

    if status[x] == "UNTESTED":

      cprint(status[x].ljust(10) + " - " + sys.argv[x].ljust(25),'white','on_blue')

    if status[x].startswith("OK"):

      cprint(status[x].ljust(10) + " - " + sys.argv[x].ljust(25),'white','on_green')

    if status[x] == "FAILED":

      cprint(status[x].ljust(10) + " - " + sys.argv[x].ljust(25),'white','on_red')

    x += 1

  print("\nWebsite to check next:\n" + str(sys.argv[chkNext]))


  status[chkNext] = webCheck(sys.argv[chkNext])

  chkNext += 1

  if chkNext >= len(sys.argv):

    chkNext = 1

Wednesday, 2 September 2020

Python 3: Basic Test for Web Site Availability

  In Python 3, doing a basic test for web site availability is very straightforward:

  • >>> import urllib.request
  • >>> print(urllib.request.urlopen("").getcode())
  • 200
  • >>>

It needs a little modification so you don't get an ugly error if the website is not available:

  • >>> try:
  • ...     a = urllib.request.urlopen("").getcode()
  • ...     print("OK (" + str(a) + ")")
  • ... except urllib.error.URLError: print("FAILED")
  • ...
  • >>>

What if we want colors?

The below shows you how to install termcolor in the command prompt. There are two methods: 1) for if 'pip download' works for you, and 2) if you have to get the file manually from one machine to install on another machine. Because I am running Python on a Windows machine, we also need to get colorama. Windows cmd doesn't support ANSI coding like linux/unix which termcolor uses (perhaps termcolor is surplus to requirements with Windows cmd...)

Method 1:

  • > python -m pip install termcolor
  • > python -m pip install colorama

Method 2:

  • python -m pip download termcolor
  • python -m pip install termcolor-1.1.0.tar.gz
  • pythom -m pip download colorama
  • python -m pip install colorama-0.4.3-py2.py3-none-any.whl

Note: See for more on termcolor.

A modified Python script using colors - save this as say

  • import sys
  • from termcolor import *
  • import colorama
  • colorama.init()
  • import urllib.request

  • url = ""
  • try:
  •     status_code = urllib.request.urlopen(url).getcode()
  •     cprint(("OK(" + str(status_code) + ")"),'white','on_green')
  • except urllib.error.URLError:
  • cprint("FAILED",'white','on_red')

Image: Example running

A very sinple example. To be enhanced in a future post...

Tuesday, 1 September 2020

NetApp OCI REST API Python Samples Requirements - On a Corporate Desktop?

From this post ‘NetApp OCI REST API and Python’ we see that it’s easy to use Python with NetApp OCI’s REST API. And this worked fine on my corporate desktop.

When I was looking in the ‘OCI REST API samples’ for Python (available for download from the OCI WebUI) - which utilizes a library to simplify access to the OCI REST API - - the pre-requisites instructions include to run (from a command prompt since my desktop is Windows):

python -m pip install --requirement requirements.txt

Where requirements.txt is:


Unfortunately, running the above generates an error on my corporate desktop. Understandably, there is some corporate restriction in place that prevents me downloading the wheel files (I count myself lucky to have access to Python in the first place!)

The error I get is:

  • Could not find a version that satisfies the requirement...
  • No matching distribution found for requests>=2.9.0 ...

But when I run the same command on my home PC, it works and downloads 5 files as below:

  • Downloading requests-2.24.0-py2.py3-none-any.whl (61 kB)
  • Downloading certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
  • Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
  • Downloading urllib3-1.25.10-py2.py3-none-any.whl (127 kB)
  • Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)

Since these are not massive files, could I download them and rename them as txt files, then email in?

It is possible to download all the above files individually over the internet, but much easier for me to just use ‘pip download’ from my home PC, as below (from CMD):

python -m pip download --requirement requirements.txt

And then we’ll convert the extensions to .txt, ZIP the files, and see if we can email into the Enterprise.

Miracle of miracles, the ZIP arrives. I unzip and convert the extensions back to .whl files!

Image: NetApp OCI Python Requirements

Now to install them (again from CMD)!

Using trial and error, below is the correct order to install them:

Note: If you try to install in the order above, you’ll find some missing requirements, and see the same error as above. 

  • python -m pip install idna-2.10-py2.py3-none-any.whl
  • python -m pip install certifi-2020.6.20-py2.py3-none-any.whl
  • python -m pip install urllib3-1.25.10-py2.py3-none-any.whl
  • python -m pip install chardet-3.0.4-py2.py3-none-any.whl
  • python -m pip install requests-2.24.0-py2.py3-none-any.whl

YES! JOB DONE! Now to utilize the OCI REST API Python examples!

To use the NetApp OCI Python REST API examples, the first thing you'll need to do from you command line is>

python install

Then in your Python shell/code, you'll want:

from oci_rest import *

Note: The above imports everything. Best practice is to be more selective.