Performing an Extract of Data from NetApp Cloud Insights Data Warehouse

Previously I wrote this post - Performing an ETL of Data from NetApp Cloud Insights. Really, it was just the E part of ETL (the extract.) And I covered using Postman and using the Linux command line. In this post we look at doing the same in Python.

Note: There is a netapp_cloudinsights.py python module, which I will explore at a later date, for this post we just use the Python basics.

Windows Command Line Pre-Setup

Firstly, if you're running this from an Enterprise desktop, you'll probably have a web proxy, so you'll need to run something like this in the DOS command prompt before running python:

set https_proxy=http://USERNAME:PASSWORD@YOURPROXY:PORT

The run python:

python

Python Command Line Instructions

Using python as a command line:

## PYTHON SETUP ##
import requests
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

## YOUR SETUP VARIABLES ##
apiKey = "YOURAPIKEY"
yourTenant = "https://YOURTENANT.cloudinsights.netapp.com"

## BASE URL and HEADERS ##
baseUrl = yourTenant + "/rest/v1"
headers = {'X-CloudInsights-ApiKey': '{key}'.format(key=apiKey)}

## EXAMPLE: select * from dwh_inventory.storage ##
api = "/dwh-management/odata/dwh_inventory/storage"

## THE REQUEST ##
j = requests.get(baseUrl + api,headers=headers,verify=False).json()

## GET THE FIRST ITEM ##
j['value'][0]

## EXAMPLE: Get name for all items in dwh_inventory.storage ##
for o in j['value']: print(o['name'])

Appendix - $top, $skip and $count

• The maximum limit is 10'000 rows by default.
• Without limit, the query will return 1000 rows.
• You can use $count to count the number of rows.
• You can use $top to select the number of rows you want.
• You can use $skip to skip rows you don't want.