Thursday, 29 June 2017

Mapping out OCI 7.3’s GUIs Main Menus and Buttons

In the following post we explore some of OnCommand Insight Server’s two GUIs (the Web UI and Java UI), graphically mapping out the main menus and buttons into a couple of images (with follow on listings) - click on the images to make them bigger. We’ll look at the Data Warehouse GUIs in a future post.

An installation of OnCommand Insight 7.3 Server with default settings configures these ports:

80: OnCommand Insight Portal Port (HTTP)
443: OnCommand Insight Portal Port (HTTP)
3306: Internal Database Port (SQL)
9310: Elasticsearch Transport Port
9200: Elasticsearch REST Port

1) Web Interface (Web UI)

Access via a web browser to:

https://FQDN_of_OCI_SERVER

Web UI Main Toolbar Menus and Buttons

Image: OnCommand Insight 7.3 Server Web UI Main Toolbar Menus and Buttons

The Toolbar

The Toolbar has the following buttons and menus:

Dashboards
- Assets Dashboard
- Violations Dashboard
- Show all dashboards (2)
- + New dashboard

Queries
- Violin Memory storage tier
- FAS3200 storage tier
- FAS3100 storage tier
- Show all queries (21)
- + New Query

Manage
- Annotations
- Annotation rules
- Applications
- Business entities
- Device resolution
- Performance policies

Admin
- Data sources
- Patches
- Acquisition units
- Ethernet monitoring units
- Health
- Troubleshooting
- Setup
- Notifications
- Audit

The Top-Right Corner

The top-right corner has the following buttons and menus:

Search
- Search assets

Help
- Help topics
- Data source support matrix
- REST API Documentation
- Check for updates
- Support
- About

Launch Java UI
Launch Reporting Portal

Logged in User
- Changed password
- Logout

2) Java Interface

Access using the desktop shortcut which points to the below target, or access via the button in the WebUI:

{OCI_INSTALL_LOCATION}\SANscreen\java\bin\javaws.exe -J https://{FQDN_of_OCI_SERVER}:443/client/app/http-client.jnlp

Java Interface Main Toolbar Menus and Buttons

Image: OnCommand Insight 7.3 Server Java Interface Main Toolbar Menus and Buttons

The Dropdown

The dropdown at the left of the menu bar has 3 menu options, each linking to a submenu with more options:

Inventory
- Hosts
- Virtual Machines
- Datastores
- NPV Switches
- Paths
- Switches
- Zones
- Storage Arrays
- Tapes
- Generic Devices

Assurance
- Changes
- SAN Path Policies
- Vulnerabilities
- Violations Browser
- Reservation Violations
- Policies
- SAN Path Violations
- Port Balance Violations

Planning
- Requests
- Plans
- Pending Tasks
- Migrations

Left Menu Bar

The left part of the menu bar (to the right of the dropdown) has 6 items:

File
- Export
- Auto Refresh
- Exit

Edit
- Select All
- Copy
- Find in This View...

View
- Progress
- Show Dashboard
- Save Profile...
- Set Profile >

Policy
- Global Policies
- FC Global Policy
- Host Virtualization Policy
- iSCSI Global Policy
- Reservations Policy

Tools
- Analyze Missing Path
- Analyze Zone and Mask
- Settings

Help
- Help Topics
- About

Right Menu Bar

The right-side of the menu bar has 2 buttons and a search entry box:

Click this button to get help
Open the Web UI
Search

Wednesday, 28 June 2017

Resetting OnCommand Insight 7.3 DWH admin Credentials the MySQL Way

The Scenario

Lost credentials to log into OnCommand Insight DWH - cannot login to either:

https://OCI_DWH_FQDN/dwh
https://OCI_DWH_FQDN/p2pd/servlet/dispatch

Image: Cannot login to OnCommand Insight DWH

The Story

After restoring a historical database into my OnCommand Insight 7.3 Data Warehouse, I managed to overwrite the admin user password with something that I didn’t know (same too the oadmin password.) It took a while to twig what had happened. I did some troubleshooting, twigged what the issue was, and then set about resolving the issue (without recourse to use tech support since this was just a lab system.)

Initial Troubleshooting

Are all the services running? YES

 Status    Service Name
 Started   MySQL
 Started   SANscreen Server
 Started   IBM Cognos*
 Started   IBM Cognos Content Database*

*At this stage we’re not really concerned about IBM Cognos services.

Can I login with DbVisualiser? YES


What do the logs say?

SANscreenAP.log @ {OCI_INSTALL_PATH}\SANscreen\cognos\c10_64\logs

The SANscreenAP.log contains login and role assignment entries (for Cognos).
When trying to log into the IBM Cognos Reporting Engine (https://OCI_DWH_FQDN/p2pd/servlet/dispatch), I was seeing:

Default Executor-thread-153 2017-06-27 10:05:00,613  [ERROR com.netapp.sanscreen.authentication.SANscreenAP] - The provided credentials are invalid:Server returned HTTP_UNAUTHORIZED for user admin

wildfly.log @ {OCI_INSTALL_PATH}\SANscreen\wildfly\standalone\log

Jboss/Wildfly is the Java app server that is ‘SANscreen Server’. No smoking gun was found in this log.

Then it twigged - the passwords had been overwritten when I restored a historical database!

Resetting SANscreen admin Credentials via MySQL

(1) Stop ‘SANscreen Server’>

net stop "SANscreen Server"

Wait for it to stop!

(2) Stop MySQL>

net stop MySQL

Wait for it to stop!

(3) In ‘Windows Explorer’ navigate to {OCI_INSTALL_PATH}\SANscreen\mysql

SHIFT + Right-click the bin folder and select ‘Open command window here’

(4) Execute the following command in the command prompt

mysqld.exe -u root --skip-grant-tables

and leave the command prompt window as it is.

Image: The First Command Prompt (this will run until we close the window)

(5) Open a new command prompt in {OCI_INSTALL_PATH}\SANscreen\mysql\bin and run>

mysql

Image: The Second Command Prompt runs mysql (MySQL 5.7.17)

(6) To show databases type>

show databases;

(7) To switch to the SANscreen database type>

use sanscreen;

(8) To show the tables type>

show tables;

(9) We’re interested in the user table. To show the columns>

show columns from sanscreen.user;

 (10) To show the username and passwords (encrypted)>

select username,passwd from sanscreen.user;

(11) I’m going to reset the admin password back to ‘admin123’. I happened to check on another install and see the passwd for ‘admin123’ = +GW1NiOxIf007lQmx5Llwzr4wic=

UPDATE user SET passwd = '+GW1NiOxIf007lQmx5Llwzr4wic=' WHERE username = 'admin';

(12) Close the first command prompt with the X in the top-right corner.

(13) In the second command prompt type>

exit;

(14) Next I had to reboot the OCI Data Warehouse server.

(15) When everything’s back up, try logging in as username = admin, password = admin123.

THE END

Credits


Monday, 26 June 2017

Connecting the Data Warehouse (DWH) to Insight Server and DWH DB Restore

Really this is part 4 to the previous 3 posts...

1) Connecting the Data Warehouse to Insight Server

1.1) Point a web browser at -

https://DWH_FQDN/dwh

- and login with admin/admin123

Image: Logging in to the Data Warehouse Portal

1.2) On Connectors click New

Image: Adding a New Connector

1.3) Enter the details for the New Connector, click Test to verify the connection, then click Save.

Note: User name = inventory, and password = sanscreen.

Image: New Connector details

Image: Successful test!

2) Data Warehouse Database Restore

Still logged into -

https://DWH_FQDN/dwh

- click on Backup/Restore in the left hand pane

Click on Browse and browse to the database backup ZIP.

Leave ‘Restore database’ and ‘Restore reports’ ticked.

Click on Restore

Note: The restore might take some time.

Image: Restoring a Data Warehouse Database and Reports

All being well, you should get ‘Restore Successful’.
And, if you log into the IBM Cognos Reporting Engine at -
https://DWH_FQDN:9300/p2pd
- the Launch dropdown should give the link to Report Studio.

Image: Click Report Studio to start generating OnCommand Insight reports!

Installation Walkthrough: OCI 7.3 and the Data Warehouse - Part 3 of 3

Part 3: Installing OnCommand Insight 7.3 Data Warehouse

Installation Pre-requisites

- The pagefile should be > 8GB.

Before the Setup Wizard

1) Open an Administrative Command Prompt

Image: Getting an Administrative Command Prompt

2) Navigate to where the EXE is stored, and from the prompt run> SANscreenDWH-x64-7.3.0-1032.exe

Note: The Setup Wizard takes a while to start because - as seen in the file properties description - this is a ‘7z Setup SFX small file’ (in other words - archive type EXE.)

Setup Wizard

1) Welcome to the OnCommand Insight 7.3.0 Setup Wizard

Click Next

2) End-User License Agreement

Tick ‘I accept the terms in the License Agreement’
Click Next

3) Customer Information

Input: Customer Name
Input: Site Name
Browse to desired installation location.
Click Next

Image: OCI 7.3.0 DWH Setup - Customer Information

4) Configure OnCommand Insight Reporting

Insight Plan DWH Portal Port (HTTP): 80
Insight Plan DWH Portal Port (HTTPS): 443
Internal Database Port (SQL): 3306
Database Size (MB): 2000
Click Next

Image: OCI 7.3.0 DWH Setup - Configure OnCommand Insight Reporting (defaults)

5) Configure OnCommand Insight Reporting (2)

Reporting Engine Port: 9300
Click Next

Image: OCI 7.3.0 DWH Setup - Configure OnCommand Insight Reporting (2) (defaults)

6) Ready to install OnCommand Insight 7.3.0

Click Install

7) Completed the OnCommand Insight 7.3.0 Setup Wizard

Click Finish

Result

All being well, you should have the following links in:
C:\ProgramData\Microsoft\Windows\Start Menu\Programs\NetApp OnCommand

IBM Cognos Business Intelligence -> O:\OCIDWH\SANscreen\cognos\c10_64\webcontent\documentation\bisrvr_mtoc.html
IBM Cognos Configuration -> O:\OCIDWH\SANscreen\cognos\c10_64\bin64\cogconfigw.exe
Start OnCommand Insight Server
Stop OnCommand Insight Server

Note: My install path was O:\OCIDWH

And the following services should be started:
- IBM Cognos
- IBM Cognos Content Database
- MySQL
- SANscreen Server = The OnCommand Insight DWH server

Logging In

Web UI: Connect to https://FQDN:9300/p2pd

User name: admin
Password: admin123

Initially, without any data, the Launch dropdown will not have a link to ‘Report Studio’ (which is the OCI reporting main tool.)

*Can also login as oadmin / ******

Next Steps to Configure the Data Warehouse

From the ‘OnCommand Insight 7.3 Data Warehouse Administration Guide’ and page 11 ‘Getting started with Data Warehouse’:

- Connecting Data Warehouse to OnCommand Insight Servers
- Building the database from history
- Setting up backup and restore processes

Installation Walkthrough: OCI 7.3 and the Data Warehouse - Part 2 of 3

Part 2: Installing OnCommand Insight 7.3 Server

Installation Pre-requisites

- The pagefile should be > 8GB.

Before the Setup Wizard

1) Open an Administrative Command Prompt

Image: Getting an Administrative Command Prompt

2) Navigate to where the MSI is stored, and from the prompt run> SANscreenServer-x64-7.3.0-1032.msi

Setup Wizard

1) Welcome to the OnCommand Insight 7.3.0 Setup Wizard

Click Next

2) End-User License Agreement

Tick ‘I accept the terms in the License Agreement’
Click Next

3) Customer Information

Input: Customer Name
Input: Site Name
Browse to desired installation location.
Click Next

Image: OCI 7.3.0 Setup - Customer Information

4) Customer Information - Configure NetApp ASUP

Configure NetApp ASUP as required:
Database: Backup without Performance Data / No Database Backup / Backup with Performance Data
Send Method: HTTP / HTTPS / FTP / Email
Logs: Extended Logs / Base Logs / No Logs
Click Next

Image: OCI 7.3.0 Setup - Configure NetApp ASUP (defaults)

5) Configure Server

OnCommand Insight Portal Port (HTTP): 80
OnCommand Insight Portal Port (HTTPS): 443
Internal Database Port (SQL): 3306
Click Next

Image: OCI 7.3.0 Setup - Configure Server (defaults)

6) Configure Elasticsearch

Elasticsearch Transport Port: 9310
Elasticsearch REST Port: 9200
Click Next

Image: OCI 7.3.0 Setup - Configure Elasticsearch (defaults)

7) Ready to install OnCommand Insight 7.3.0

Click Install

8) Completed the OnCommand Insight 7.3.0 Setup Wizard

Click Finish

Image: OCI 7.3.0 Setup - Completed

Result

All being well, you should have a desktop shortcut for ‘OnCommand Insight Client’ that points to something like the below:
O:\OCISVR\SANscreen\java\bin\javaws.exe -J https://OCISVR.lab.priv:443/client/app/http-client.jnlp

The following services should be started (check in services.msc):
- Elasticsearch
- MySQL
- SANscreen Acq
- SANscreen Server

Logging in

Java UI: Double-click the link on the desktop, accept any Java prompts, and default credentials are:

User name: admin
Password: admin123

Web UI: Connect to https://FQDN:443

*Can also login as oadmin / ******

Licensing

From the Web UI at https://FQDN/uiserver/#admin/settings/license
Click on the Update License button
Copy and paste in your license
Click Save

Next Steps to Configure OCI

From the ‘OnCommand Insight 7.3 Configuration and Administration Guide’ and page 11 ‘Setting up Insight’:

- Install your Insight licenses.
- Set up your data sources in Insight.
- Set up user accounts.
- Configure your email.
- Define your SNMP, email, or syslog notifications if needed.
- Enable automatic weekly backups of your Insight database.
- Perform any advanced configuration steps required, including defining annotations and thresholds.

Installation Walkthrough: OCI 7.3 and the Data Warehouse - Part 1 of 3

In the following posts we install NetApp OnCommand Insight (OCI) 7.3 and the Data Warehouse on two separate Windows Servers. The main purpose is to document the installation prompts and required inputs.

Note: I am using 2008 R2 Enterprise SP1 servers here. Windows Server 2008 R2 SP1 does not go end of extended support with Microsoft until 14th January 2020 (see here). If my lab had 2012 R2 I would have used this, but - for purposes of illustrating the install - it doesn’t matter.

Part 1: Getting the Softwares

1) Check your Configuration in the NetApp Interoperability Matrix Tool


OnCommand Insight 7.3.0 supported with
... Host OS includes = Microsoft Windows Server 2008 R2 EE SP1
... Browsers = Google Chrome 46 to 55, IE 10 and IE 11, Firefox 40 to 51

2) Check resource requirements from the OCI 7.3 Documentation


2.1) Insight Server requirements (page 11)

Memory and CPU: “For environments with up to 100 storage arrays, 5000 Fibre Channel switch ports, and 5000 virtual machines, use an 8 core 32 GB memory server. This is a general guideline...” Note: Less is acceptable for a lab setup (I went with 4 core and 16GB RAM.)

Available disk space: “500gb recommended for typical environment (100gb on OCI install drive)...”

2.2) Data Warehouse and Reporting server requirements (page 13)

CPU: 8 CPU core”
Memory:40 GB RAM...”
Available disk space: 100 GB. Installation requires 20 GB free on the C: drive.”

Note: Less is acceptable for a lab setup (I went with 4 core and 16GB RAM.)

3) Download the Installers


There are 5 different installers. For this walkthrough we’re only interested in the highlighted two:


4) End of Part 1

All being well, we should have two suitable servers and the following softwares downloaded:

SANscreenServer-x64-7.3.0-1032.msi
SANscreenDWH-x64-7.3.0-1032.exe

Image: OCI Server and DWH softwares downloaded

Sunday, 25 June 2017

Exploring the OCI 7.3 Data Warehouse Data Marts with DbVisualizer: Part 2 of 2

... continuing from part 1.

Part 2: Exploring the OCI 7.3 Data Warehouse Database

DbVisualiser displays the DWH Data Marts as listed below:

- dwh_capacity
- dwh_capacity_efficiency
- dwh_custom
- dwh_fs_util
- dwh_inventory
- dwh_performance
- dwh_ports
- dwh_reports
- information_schema
- tmp

Image: Data Warehouse Data Marts in DbVisualizer

And here’s a listing of all the tables in each dwh* Data Mart, with TABLE_NAME and TABLE_TYPE.

Listing of Tables in Data Warehouse Data Marts

dwh_capacity (32 tables)

TABLE_NAME ---------------------------- TABLE_TYPE
application_dimension                         VIEW
application_group_bridge                      VIEW
application_group_dimension                   VIEW
business_entity_dimension                     VIEW
capacity_type                                 TABLE
cardinals                                     VIEW
chargeback_fact                               TABLE
datastore_dimension                           VIEW
date_dimension                                VIEW
disk_group_capacity_fact                      TABLE
disk_group_capacity_future_fact               TABLE
disk_group_dimension                          VIEW
host_dimension                                VIEW
host_group_bridge                             VIEW
host_group_dimension                          VIEW
internal_volume_capacity_fact                 TABLE
internal_volume_dimension                     VIEW
qtree_capacity_fact                           TABLE
qtree_dimension                               VIEW
service_level_dimension                       VIEW
sp_time_to_fill                               TABLE
storage_and_storage_pool_capacity_fact        TABLE
storage_and_storage_pool_capacity_future_fact TABLE
storage_dimension                             VIEW
storage_pool_dimension                        VIEW
tier_dimension                                VIEW
vm_capacity_fact                              TABLE
vm_dimension                                  VIEW
volume_capacity_fact                          VIEW
volume_dimension                              VIEW
volume_fact_current                           TABLE
volume_history_capacity_fact                  TABLE

dwh_capacity_efficiency (3 tables)

TABLE_NAME ---------------------------- TABLE_TYPE
date_dimension                                VIEW
efficiency_fact                               TABLE
storage_dimension                             VIEW

dwh_custom (1 table)

TABLE_NAME ---------------------------- TABLE_TYPE
esx_subscription_history_fact                 TABLE

dwh_fs_util (8 tables)

TABLE_NAME ---------------------------- TABLE_TYPE
compute_resource_dimension                    VIEW
date_dimension                                VIEW
file_system_dimension                         VIEW
fs_util_fact                                  TABLE
host_dimension                                VIEW
storage_dimension                             VIEW
tier_dimension                                VIEW
vm_dimension                                  VIEW

dwh_inventory (99 tables)

TABLE_NAME ---------------------------- TABLE_TYPE
acq_acquisition_unit                          TABLE
acq_data_source                               TABLE
annotation_value                              TABLE
application                                   TABLE
auto_tiering_policy                           TABLE
auto_tiering_policy_constraint                TABLE
backend_lun                                   TABLE
backend_lun_to_volume                         TABLE
backend_path                                  TABLE
business_entity                               TABLE
cm_lun_requests                               VIEW
cm_lun_requirement                            TABLE
cm_port_requests                              VIEW
cm_port_requirement                           TABLE
cm_request                                    TABLE
compute_resource                              TABLE
disk                                          TABLE
disk_group                                    TABLE
disk_group_to_disk                            VIEW
disk_group_to_storage_pool                    VIEW
dr_internal_volume_replica                    TABLE
dr_qtree_replica                              TABLE
dr_replica                                    VIEW
dr_volume_replica                             TABLE
fabric                                        TABLE
fc_name_server_entry                          TABLE
file_system                                   TABLE
file_system_to_virtual_disk                   TABLE
file_system_to_volume                         TABLE
generic_device                                TABLE
generic_device_port                           TABLE
host                                          TABLE
host_adapter                                  TABLE
host_port                                     TABLE
host_to_application                           TABLE
hv_data_store                                 TABLE
hv_data_store_to_internal_volume              TABLE
hv_data_store_to_volume                       TABLE
hv_datastore_to_nas_share                     TABLE
hv_host_to_nas_share                          TABLE
hv_lun                                        TABLE
hv_lun_to_volume                              TABLE
hv_nas_share                                  TABLE
hv_nas_share_to_share                         TABLE
hv_path                                       TABLE
hv_server                                     TABLE
hv_virtual_disk                               TABLE
hv_virtual_disk_to_volume                     TABLE
hv_virtual_machine                            TABLE
hv_virtual_machine_to_disk                    TABLE
internal_volume                               TABLE
internal_volume_to_application                TABLE
iscsi_logical                                 TABLE
license_details                               TABLE
logical                                       TABLE
nas_cr_logical                                TABLE
nas_file_share                                TABLE
nas_logical                                   VIEW
nas_share                                     TABLE
nas_share_initiator                           TABLE
object_to_annotation                          TABLE
physical_path                                 TABLE
port_connectivity                             TABLE
qtree                                         TABLE
qtree_to_application                          TABLE
quota                                         TABLE
service_level                                 VIEW
share_to_application                          TABLE
storage                                       TABLE
storage_controller                            TABLE
storage_node                                  TABLE
storage_node_to_internal_volume               VIEW
storage_node_to_port                          VIEW
storage_node_to_storage_pool                  VIEW
storage_node_to_volume                        VIEW
storage_pool                                  TABLE
storage_pool_to_backend_lun                   VIEW
storage_pool_to_disk                          VIEW
storage_port                                  TABLE
storage_to_application                        TABLE
switch                                        TABLE
switch_port                                   TABLE
switch_port_to_application                    TABLE
switch_to_application                         TABLE
tape                                          TABLE
tape_controller                               TABLE
tape_port                                     TABLE
tier                                          VIEW
violation                                     TABLE
virtual_switch                                TABLE
virtual_to_backend_volume                     VIEW
vm_to_application                             TABLE
volume                                        TABLE
volume_in_storage_pool                        VIEW
volume_map                                    TABLE
volume_mask                                   TABLE
volume_to_application                         TABLE
zone                                          TABLE
zone_member                                   TABLE

dwh_performance (46 tables)

TABLE_NAME ---------------------------- TABLE_TYPE
application_dimension                         VIEW
application_group_bridge                      VIEW
application_group_dimension                   VIEW
application_volume_hourly_performance_fact    TABLE
business_entity_dimension                     VIEW
capacity_volume_fact_current                  VIEW
connected_device_dimension                    VIEW
datastore_dimension                           VIEW
date_dimension                                VIEW
disk_daily_performance_fact                   TABLE
disk_dimension                                VIEW
disk_hourly_performance_fact                  TABLE
fabric_dimension                              VIEW
host_dimension                                VIEW
host_group_bridge                             VIEW
host_group_dimension                          VIEW
host_vm_daily_performance_fact                TABLE
host_vm_hourly_performance_fact               TABLE
host_volume_hourly_performance_fact           TABLE
internal_volume_daily_performance_fact        TABLE
internal_volume_dimension                     VIEW
internal_volume_hourly_performance_fact       TABLE
port_dimension                                VIEW
service_level_dimension                       VIEW
storage_dimension                             VIEW
storage_node_daily_performance_fact           TABLE
storage_node_dimension                        VIEW
storage_node_hourly_performance_fact          TABLE
storage_pool_dimension                        VIEW
switch_dimension                              VIEW
switch_performance_for_host_hourly_fact       TABLE
switch_performance_for_port_hourly_fact       TABLE
switch_performance_for_storage_hourly_fact    TABLE
switch_performance_for_tape_hourly_fact       TABLE
tape_dimension                                VIEW
tier_dimension                                VIEW
time_dimension                                TABLE
vm_daily_performance_fact                     TABLE
vm_dimension                                  VIEW
vm_hourly_performance_fact                    TABLE
vmdk_daily_performance_fact                   TABLE
vmdk_dimension                                VIEW
vmdk_hourly_performance_fact                  TABLE
volume_daily_performance_fact                 TABLE
volume_dimension                              VIEW
volume_hourly_performance_fact                TABLE

dwh_ports (7 tables)

TABLE_NAME ---------------------------- TABLE_TYPE
cardinals                                     VIEW
connected_device_dimension                    TABLE
date_dimension                                VIEW
port_dimension                                VIEW
ports_fact                                    TABLE
ports_fact_current                            VIEW
switch_dimension                              VIEW

dwh_reports (2 tables)

TABLE_NAME ---------------------------- TABLE_TYPE
report_orphaned_volumes                       VIEW
report_storage_summary_aux                    VIEW

THE END