Monday, 27 August 2012

Veeam nworks Management Pack 5.7 - How to Troubleshoot


Credit: the following information is strongly based on the Veeam ProPartner University “Veeam nworks MP 5.7 - Troubleshooting” course and updated for SCOM 2012 with a new selection of images.

1. Collector and EM server not displayed in OpsMgr

Sympton:
Collector and EM server are not displayed in:
Operations Console > Monitoring > nworks VMware > Enterprise Manager > _Enterprise Manager Server State
Operations Console > Monitoring > nworks VMware > Enterprise Manager > Collector Servers > _Collector State



Probable causes:
·         Uncompleted discovery process
·         Insufficient permissions of the OpsMgr Agent Action Account
·         Communication problems between OpsMgr agent and OpsMgr server

Solution:
·         Wait for discovery to complete - or -
·         Restart the OpsMgr agent, or System Center Management service, on the Collector server or Enterprise Manager server to initiate discovery


·         Check that the proper permissions are assigned to the OpsMgr Agent Action Account (recommended that SCOM Agent Local System Account permissions are used - must have full administrative access to the local server)

Check:
Operations Console > Administrator > Agent Managed > find the server hosting the Collector and/or Enterprise Manager, and check if the Operations Manager Agent has been deployed on that server and if it is reporting back with Green status



2. Collector and EM server cannot connect to VMware Host/vCenter Server

Probable causes:
·         Incorrect access credentials for the nworks service account
·         Firewall or network issues
·         DNS configuration errors

Solution 1:
Nworks Enterprise Manager Web UI > Select the Host/vCenter in question > Operations: ‘Test Connection’ button
Note: There is also a ‘Test All Connections’ button



From the Host/vCenter > Administration > Edit Connection > verify the Connection settings
Note: At least read-only role across a vCenter server is required

Solution 2:
Check that the traffic on the port used for API access to the VMware systems is not blocked (default 443.) Either open firewall for the required port, or configure the ‘Proxy settings’ in nworks Enterprise Manager Connection settings



Solution 3:
If no connection using an FQDN DNS name, try using IP address.

3. VMware objects do not appear in OpsMgr views

Sympton:
VMware objects do not appear in:
Operations Console > Monitoring > nworks VMware > _All VMware State
Note: This view includes _Cluster State, _ESX Host State, _VM State

Probable cause:
Proxying is not enabled for the Operations Manager agent on the Collector Server(s)

Solution 1:
Operations Console > Administration > Device Management > Agent Managed > Agent Properties > Security tab > verify “Allow this agent to act as a proxy and discover managed objects on other computers” is ticked



Solution 2:
If only certain VMware objects are missing or have not been updated:
Nworks Enterprise Manager Web UI > Enterprise Manager tab > Operations > Rebuild Full Topology
Note: This will take several collection cycles (default 5 minutes each) to complete



4. Unstable behaviour of the OpsMgr Health Service

Symptons:
·         Heartbeat failure
·         OpsMgr agent restarts
·         Gaps in reports/charts
·         Agent and/or VMware objects “grayed out”

Probable causes:
·         OpsMgr Health Service is overloaded
·         Incorrect agent configuration
·         Collector is overloaded

Solution 1:
1.       Remove nworks monitoring jobs from the Collector
2.       Stop the “System Center Management” service on the Collector server
3.       Delete the “Health Service State” folder at C:\Program Files\System Center Operations Manager\Agent
Note: The Health Service State folder is recreated when the “System Center Management” service is restarted
4.       Start the “System Center Management” service
5.       Observe the “Operations Manager” log in Event Viewer > Applications and Services Logs
6.       Reapply nworks monitoring jobs to the Collector

Solutions 2:
·         Run Configure Operations Manager Agent task (include in nworks MP) to reapply configuration to the agent
·         Relocate monitoring jobs to other Collectors
·         Apply tuning options - such as increasing collection intervals, or disabling certain data classes - to reduce the amount of data processed by the Collector

5. No events/alerts in OpsMgr console

Sympton:
Alerts on vCenter events are not received in the Operations Manager console

Probable causes:
·         Connection or configuration issue between Enterprise Manager/Collector server and VMware systems
·         Disabled Rules and/or Monitors on the OpsMgr
·         Modified thresholds for alerts on vCenter events

6. Performance data is not collected

Sympton:
No data in OpsMgr console

Probable cause:
Windows Event Logging facility is not working properly; that is - events are not being delivered to the custom “nworksEventLog” found in Event Viewer on both Enterprise Manager and Collector(s)



Solution:
·         Check the existence of the ”nworksEventLog” and that new events are being delivered (at the very least, EventID 117 should appear at 5-minute intervals)
·         If the log does not exist: reboot the system, open the Computer Management snap-in, check nworks Virtual Infrastructure Collector service is installed and running, and that the nworksEventLog has been created
·         If the above does not help: reinstall the collector
·         If log still not created: check the license in the nworks Enterprise Manager Dashboard - the license type must be ‘MOM/SCOM’



7. Performance data is not delivered

Sympton:
No data in OpsMgr performance graphs

Probable cause:
Windows Management Instrumentation (WMI) is not working properly

Solution (using the Windows Management Instrumentation Tester):
·         From the command prompt> type wbemtest
·         Click Connect…
·         Enter for Namespace root\nworks
·         Click Connect



·         Click Enum Instances…
·         Enter superclass name as Host Stats (this an example metric class for collection from ESX hosts)



·         The presence of instances confirms the Collector is posting collected host metrics into WMI for OpsMgr agent



·         If no instances shown or they do not update every 5 minutes, there is an issue with connection to vCenter. Check connection as above, and if okay then try re-installing the Collector.

How to install Veeam nworks MP for VMware with SCOM 2012: Part 6/6 - Configuring Custom Overrides

Parts in this series:

Part 6 - Configuring Custom Overrides - Example

Note: The recommended way to apply overrides for nworks MP is by using the nworks MP Resource Kit from the Veeam website which includes the nworks.VMware.2007R2RequiredOverrides.xml management pack which automatically applies recommended overrides.

Part 6.1 - Monitors

System Center 2012 Operations Manager - Operations Console > Authoring > Management Pack Objects > Monitors



Locate Health Service > Entity Health > Performance > System Center Management Health Service Performance > System Center Management Health Service Memory Utilization > Health Service Private Bytes Threshold



Right-click and select Properties > then from the ‘Health Service Private Bytes Threshold Properties’ select the Overrides tab > click the Override… button and choose ‘For a group…’



Find ‘nworks VMware Collector Servers’ and click OK



Check the Override box by ‘Agent Performance Monitor Type - Threshold’ and change the ‘Override Value’ (scroll to the right) to 2712345678



Then underneath ‘Select destination management pack:’ select ‘New…’



Call it something like ‘nworks overrides MP custom’ and click Next > and then click Create



And with the destination management pack selected as the newly created custom management pack > Click Apply > and then Click OK



Part 6.2 - Rules

Note: This is an example for illustration purposes only.

System Center 2012 Operations Manager - Operations Console > Authoring > Management Pack Objects > Rules



Locate Type: Agent > Collect System\System Uptime



Right-click and select Properties > then select the Overrides tab > click the Override… button and choose ‘For a group…’



Find ‘nworks VMware Collector Servers’ and click OK



Check the Override box and change the Override Value to 23456



And with the destination management pack selected as the previously created ‘nworks overrides MP custom’ management pack > Click Apply > and then Click OK


How to install Veeam nworks MP for VMware with SCOM 2012: Part 5/6 - Import nworks Management Pack

Parts in this series:

Part 5 - Import nworks Management Pack and Further Configuration in System Center 2012 Operations Manager

System Center 2012 Operations Manager - Operations Console > Administration > Management Packs > Right-click and choose ‘Import Management Packs…’



Import Management Packs: Select Management Packs > Add > Add from disk … > No to the “Would you like to search the online catalog …?” > and locate your nworks.VMware.VEM.mp management pack file > click Install



After import completes with status = ‘Imported’, click Close



Monitoring > nworks VMware > Enterprise Manager > Collector Servers > _Collector State > select the Collector

And from the right-hand pane click “Configure OpsMgr agent”, Run the task, and click Close when complete:
Note: This will need to be done for every collector and all new collectors!



Next we import the nworks.VMware.2007R2RequiredOverrides.xml Management Pack from the Veeam mp_resource_kit_5.7.0.1054 similarly to the above process:
Note: The 2007R2 one is correct for SCOM 2012



Finally we import the nworks.VMware.BPAC.xml Management Pack from the Veeam nworks.vmware.bpac:



And if we look at the list of installed Management Packs, we see:


How to install Veeam nworks MP for VMware with SCOM 2012: Part 4/6 - Install nworks Collector Server

Parts in this series:

Part 4 - Install nworks Collector Server

Memory Requirements: 3GB minimum; and x64 OS with 6GB RAM recommended.
CPU Requirements: 2 x 2GHz minimum recommended.

Server Manager > Features > Add Features > Select ‘.NET Framework 3.5.1’ and follow the wizard to install



Double-click on the nworksVIC.exe executable:



Click Next >



Accept the License Agreement and Next >

Destination Folder: Accept/Change… install path and Next >



Enterprise Manager Server Information: Enter EMS server FQDN name, and port (default is 8084) and Next >
Note: Any problems connecting here - check the Windows Firewall on the EMS!



Service Credentials: Enter nworks Virtual Infrastructure Collector Service credentials (needs to be local admin and member of nworks Enterprise Manager accounts local group on the EMS server) and Next >
Note: In this lab we’re using the domain administrator account, in practice would use a different account.



Ready to Install the Program: Install



InstallShield Wizard Completed: Finish



Note: The collector will appear in Veeam nworks Enterprise manager



Repeat for however many collectors that need to be installed!

How to install Veeam nworks MP for VMware with SCOM 2012: Part 3/6 - Install nworks Enterprise Manager Web UI

Parts in this series:

Part 3 - Install nworks Enterprise Manager Web UI

Note: The UI can be installed on the EMS or a different computer, and here we are installing on the EMS (the UI is not supported on a VMware vCenter server)

Server Manager > Roles > Add Roles > Select ‘Web Server (IIS)’ and follow the wizard to install



Additional role services required for Web Server (IIS) are:
·         IIS 6 Management Compatibility
·         Windows Authentication support
·         ASP. NET

Double-click on the nworksUI.exe executable:



Click Next >



Accept the License Agreement and Next >

Destination Folder: Accept/Change… install path and Next >



Enterprise Manager Server Information: Enter EMS server FQDN name, and port (default is 8084) and Next >



nworks Enterprise Manager WebUI Web Site Configuration: accept/change default port 4430 and Next >



Ready to Install the Program: Install



InstallShield Wizard Completed: Finish



Log off and log back in.

Note 1: nworks Enterprise Manager should now be accessible on http://{Name_or_FQDN_of_EM_WebUI}:4430
Note 2: There is a local group called “nworks Enterprise Manager accounts” on the EMS for “Allows users to access Enterprise Manager via Web UI, and remote Collectors to access Enterprise Manager for Licensing and Configuration settings.”