Register Login

Troubleshoot Grey HANA Metrics in Technical Monitoring

Updated May 18, 2018

In Transaction SM_WORKCENTER or SOLMAN_WORKCENTER. We have selected Technical Monitoring, System Monitoring and systems - Select Managed System.

It says go to System Monitoring tab "Start Embedded" or "Start new window" then Select System.

Solution:

Before creating an incident, review the following points which may be applicable for the issue:

A. There are two common potential root causes for grey metrics based on the extractor E2E_DB_ALERTS if the embedded statisticsserver is enabled:

Embedded statistics server enabled means: no statisticsserver process listed in the HANA services list and parameter: nameserver.ini -> [statisticsserver]->active=true.

Especially with HANA Revisions < 90 HANA may have disabled alerts internally because their data collection runtime was too long. Before HANA Rev 90, these alerts are not automatically enabled again and lead to grey metrics in Solman. Check the following SAP :

2131512 - HANA alerts continually disabled due to timeout

You will see status ‘Disabled’ in table _SYS_STATISTICS.STATISTICS_SCHEDULE for each alert id (column ID) which was switched off by HANA. You can also see the last execution time of the alert check per ID and if this is more than 2 times the metric frequency in Solman, you will get a grey metric. It also means that HANA internally does not check these alert conditions any more.

Missing permission for the DB user used by Solman to read the alert data. With the embedded statisticsserver two addiitional permissions (in addition to the MONITORING role) need to be granted to the DB user on HANA side.

See the very end of the following SAP :

1917938 - Migration of the statistics server for Revision 74 or higher

If the permissions are missing, you get a permission error when executing the following command in HANA studio with the DB user in question:

call _sys_statistics.statistics_schedulablewrapper_TEST('Manual',now(),30,1,?)

(this is a test call for fetching alert id 30)

B. Implementation of SAP S (Latest Versions): Implement the latest version of the following s:

2211415 - Alerting composite SAP

2008396 - ST-PI: E2E Datacollectors for HANA 

C. In the scheduling table in HANA Studio, the Alert will not deliver any values if it has status "Inactive".

To activate the alert just send an UPDATE table in the scheduling table and change the status to “Idle” with a capital I.

: if the metric "Table Consistency" is delivered as "Inactive", together with setting id =83 "Table Consistency" to "Idle", you should also set id=5047 to "Idle".

D. Grey Metric "IQ DB: DBSpace Usage"

IQ DB: DBSpace Usage (Database disk usage) only works if using Data Tiering.

E. Other reasons for grey metrics in Solman are:

There was never any information reported for that metric (this is also the case after a fresh reconfiguration until a metric was once collected).

The last reported metric information is more than 2 times older than the collection interval defined for that metric in Solman.

A metric has the threshold type “Already Rated” and was explicitly reported with the rating “Grey”=0. There are quite a number of metrics defined as “Already Rated” for HANA but a reported rating is not supposed to be grey.

F. The following reasons may cause the metric data not to be reported for 2 collection periods:

The Solution Manager collection framework has a resource problem and therefore cannot schedule all collection calls as timely as expected. When you look at the metric history for the affected metrics, you would see gaps longer than expected.

If HANA statisticsserver detects that an internal metric collection took too long (longer than the HANA internal collection period for that metric) then it temporarily disables that metric (status = ‘Disabled’). Solution Manager would definitely not get the metrics data (for metrics of type “Already Rated” and maybe also not for the others). 

Then it will turn grey but it will also report in the metric “Disabled Alerts” that HANA alerts had been disabled. HANA statisticsserver should automatically enabled this alert ids again after some time.

If a HANA alert was set to “Inactive” (some are delivered by default as inactive like alert id 83) then the metric will remain grey until it is once manually enabled (set to ‘Idle’).

There is an ABAP Enqueue Lock used in the database alert metric data collection module (E2E_DB_ALERTS) which prohibits that it runs multiple times in parallel for the same database. All metrics with the same collection frequency are fetched with a single call to this module but for different collection intervals also different calls to this module executed. If it happens now that the call for one frequency is not yet finished and the call for another frequency comes then the second one will return immediately without returning any metric data. If the same frequency’s call is affected twice in a row by that “bad luck” the metric will become grey until a successful call for that frequency happens again. Actually this affects all metrics with the same frequency together (either they can all be executed or none).

HANA itself also does regular alert checking internally independent from Solution Manager. This also takes some time for each alert id and during that time the calls from Solution Manager asking for the same alert id are blocked. In this case that specific metric will not be returned and if that happens twice in a row for the same metric it will become grey until it can be collected successfully once again.

G. Reconfigure Monitoring

The reconfiguration of the monitoring needs to be done when any changes have been performed to the system, activate the monitoring for your systems again:

In transaction Solman_Setup

Select Technical Monitoring

Select 5 Define Scope

Select System ID

Select "Next" or 6 Setup Monitoring

Select Edit

Select Apply and Activate "All Managed Objects in background"


×