Sunday, 15 May 2022

[CI][REST API][Python] Adding Annotation Values via the REST API

Work In Progress

Some more NetApp Cloud Insights REST API with Python basics here.

Again using Python like a shell, no creating and running scripts.

We want to update some annotations via the REST API. We have already got a list of annotations not found in CI using the method here: Comparing OCI vs CI Annotations and Annotation Values.

0) Setup

Notes: 1) We're just running things directly in the Python Client, no creating scripts here. 2) Update the highlighted bits as per your CI tenant.

  • import requests
  • import urllib3
  • urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
  • apiKey = "YOUR_API_KEY"
  • yourTenant = "aaXXXX.bYY-cc-Z.cloudinsights.netapp.com"
  • baseUrl = "https://" + yourTenant + "/rest/v1/"
  • headers = {'X-CloudInsights-ApiKey': '{key}'.format(key=apiKey)}
1) Get Annotation ID from the Annotation Name

This is written as a Python one liner.

a = "YOURANNOTATION";j = requests.get(baseUrl + "assets/annotations/" + a, headers=headers, verify=False).json();j['id']

Other useful stuff - 1) To get display all the values2) The count of all the values:

for v in j['enumValues']: print(v['name'])

Other useful stuff - 2) The count of all the values:

len(j['enumValues'])

Other useful stuff - 3) Finding out if the value is already in enumValues:

values = []
for v in j['enumValues']: values += [v['name']]
if 'TEST_DATA' in values: print("EXISTS!")

2) Add a Value to the Annotation you've just got the Annotation ID for

2.1) Type FLEXIBLE_ENUM (List in the UI)

The REST API to add the annotation is:

PATCH /assets/annotations/{id}

The request body needs to include all the existing values (you could remove existing values that aren't in use and no longer required) and the new value(s).

For example, for a Country annotation where you already have Argentina and Brazil, and you want to add Chile, the request body is:

{
  "name": "Country",
  "description": "",
  "enumValues": [
    {
      "name": "ARGENTINA"
      "label": "ARGENTINA"
    },
    {
      "name": "BRAZIL"
      "label": "BRAZIL"
    },
    {
      "name": "CHILE"
      "label": "CHILE"
    }
  ]
}

In Python we can use (also see here):

requests.patch(url, data=json.dumps(payload), headers=headers, verify=False)

To be continued.



[CI][DWH][ODATA] Limits $top $skip and $count

Carrying on from the previous post:

Something important I should mention about running queries against the NetApp Cloud Insights Data Warehouse is that there are certain limits on the amount of data that will be returned.

  • The maximum limit is 10'000 rows by default.
  • Without limit, the query will return 1'000 rows.
  • You can use $count to count the number of rows.
  • You can use $top to select the number of rows you want (remembering the 10'000 row limit)
  • You can use $skip to skip rows you don't want.

Playing around with Python and the dwh_inventory.hosts table (which is quite big in the environment I am running these queries on). These were my findings.

I couldn't get $count to work without returning data too. But what I could do is a count with just return 1 result.

Note: In the below I am using Python shell one liners (because it is super easy to up arrow and edit the one line of Python when you're playing around.)

Example: Getting the full count with just one row of data returned.

api = "dwh-management/odata/dwh_inventory/host?&count=true&$top=1";url = baseUrl + api;jsondata = requests.get(url, headers=headers, verify=False).json();jsondata['@odata.count']

In my case this returned 40929.
So if we want to get all the output then we need to do the following.

Example: Getting all 40929 results

hosts = []

api = "dwh-management/odata/dwh_inventory/host?&top=10000"
url = baseUrl + api
jsondata = requests.get(url, headers=headers, verify=False).json()
hosts += jsondata['value']

api = "dwh-management/odata/dwh_inventory/host?&top=10000&skip=10000"
url = baseUrl + api
jsondata = requests.get(url, headers=headers, verify=False).json()
hosts += jsondata['value']

api = "dwh-management/odata/dwh_inventory/host?&top=10000&skip=20000"
url = baseUrl + api
jsondata = requests.get(url, headers=headers, verify=False).json()
hosts += jsondata['value']

api = "dwh-management/odata/dwh_inventory/host?&top=10000&skip=30000"
url = baseUrl + api
jsondata = requests.get(url, headers=headers, verify=False).json()
hosts += jsondata['value']

api = "dwh-management/odata/dwh_inventory/host?&top=10000&skip=40000"
url = baseUrl + api
jsondata = requests.get(url, headers=headers, verify=False).json()
hosts += jsondata['value']

len(hosts)

At least that is how it should be working. I'm not sure if I was having weird issues with my Python/VDI, but what worked at one point, stopped working later on...

Note: Programmatically you would use a for loop for this, just demonstrating the top and skip here.