Monitoring

ZendHQ provides a feature allowing you to monitor your PHP application. In general, monitoring allows insight into application health, by providing notifications when certain conditions occur, such as high CPU usage, high memory usage, slow requests, and more. You may either passively retrieve these notifications by using the ZendHQ User Interface to view and get details on events, or you may setup webhooks that send notifications to you actively.

Defining Monitoring Rules

You have several options for defining monitoring rules for your application: via the ZendHQ User Interface, via a configuration file, or via the websocket API, as detailed below.

Using the GUI

To define rules in the ZendHQ User Interface, navigate first to the Configuration section under the Monitoring item of the ZendHQ User Interface. ZendHQ comes with a number of default rules demonstrating the different rule types.

General configuration

To define a new rule, select Add event rule in the upper right of the Monitoring Configuration screen. Alternately, you can click on a rule from the list of rules to edit an existing rule. Both actions open a rule configuration callout dialog.

All rules allow you to specify:

  • Rule name: Use this to provide a descriptive name for the event, such as "DBAL query errors".
  • Trigger type: This sets the type of event, and enables additional configuration options (see below for more details).
  • Callback URL: This is a webhook callback that ZendHQ calls with event details to provide active notifications or integrations with other systems (see below for more details).
  • Severity level: This is one of Notice, Warning, or Critical, and you can use it to group events, as well as quickly identify events which require action.

After configuring a rule, select Save in the lower right of the monitoring rule callout. If you wish to cancel rule creation or decide not to make changes, select Cancel. You may also choose to delete an existing rule from the rule callout dialog by selecting Delete in the lower left of the dialog.

Trigger types

When defining a monitoring rule, you select a trigger type. Types include:

  • Function Error: Triggered when an error state occurs while triggering a function. When defining a Function Error event, you can add one or more function names you wish to monitor for errors. Function names may be any PHP internal function, any user-defined function, or even a method name (in the form of {Class Name}::{Method Name}). The default rule monitors each of:
    • curl_exec
    • fopen
    • file_get_contents
    • file_put_contents
    • fsockopen
    • ftp_connect
    • ftp_ssl_connect
    • ldap_connect
    • pcntl_exec
    • socket_connect
    • stream_socket_client

  • Database Error: Triggered when a database error occurs. This is a Function Error event that monitors various database extension methods (For example. mysqli_connect, PDO::prepare, etc.).
  • Slow Function Execution: Triggered when a function takes longer than a particular threshold to complete. When configuring this type, each severity level, when selected, allows you to define an execution time threshold that triggers the event. For instance, you might trigger a warning when execution time exceeds one second, and a critical severity when it exceeds three seconds. Additionally, you can optionally provide specific function names to monitor using this rule; if no function names are provided, it monitors all function and method calls.
  • Slow Query Execution: Triggered when a database query takes longer than a particular threshold to complete. This is a Slow Function Execution event that monitors various database extension query methods (For example. mysqli_query, PDO::query, PDOStatement::fetchAll, etc.).
  • Slow Request Execution: Triggered when the duration of the request lifecycle takes longer than a particular threshold to complete. When configuring this type, each severity level, when selected, will allow you to define an execution time threshold that triggers the event. For instance, you might trigger a warning when execution time exceeds two seconds, and a critical severity when it exceeds five seconds. This differs from Slow Function Execution because it considers the entire request time, not individual functions.
  • High Memory Usage: Triggered when the memory usage during a request is above a particular threshold. When configuring this type, each severity level, when selected, allows you to define a memory usage limit threshold that triggers the event. For instance, you might trigger a warning when memory usage exceeds 24MB, and a critical severity when it exceeds 48MB.
  • PHP Error: Triggered when a PHP error occurs. When configuring this type, each severity level, when selected, allows you to define a list of PHP error types that triggers the event. For instance, you might trigger a warning when an E_WARNING, E_CORE_WARNING, E_COMPILE_WARNING, E_USER_WARNING, or E_RECOVERABLE_ERROR occurs.
  • Custom Event: Triggered when code calls the function monitor_custom_event() or monitor_custom_event_ex() (both of which are defined by the ZendHQ ZendPHP extension). This event type offers no additional configuration options.

Using a configuration file

For automation purpose, you may want to define and version your rules. The ZendHQ service reads the default rules from the /opt/zend/zendphp/etc/default_monitor_rules.json file; by supplying a different file, ZendHQ reads from that file to seed itself on initialization.

See the section on Monitoring Rules JSON Structure for more details.

If your ZendHQ instance has been previously initialized, you may need to run the mon.load_default_rules method of the websocket API in order to pick up the new defaults.

Using the websocket API

After ZendHQ has initialized, you may also use the websocket API to configure monitoring rules. After first connecting to and authenticating against the websocket, you can run the following command:

{"id":1,"method":"conf.set","params":{"ns":"monitor","key":"rules","value":[...]}}

The value is an array of 0 or more rules, as defined in the Monitoring Rules JSON Structure.

Viewing events

Whenever a monitoring rule is created, it generates an event. An event contains information collected by the monitoring rules You may view events via the Event Log section of the Monitoring navigation item.

The Event Log details the events that ZendHQ has collected. By default, each event is shown as a separate entry; if you have enabled the ZendHQ daemon's zendhqd.monitor.aggregate_events setting, you get one line per event type, with a count indicating the number of times it has triggered.

To view details on an event, click the line representing the event, which opens an event details callout. This callout shows:

  • Severity
  • A timestamp of when it was triggered. If event aggregation is enabled, and more than one event of that type was captured, it shows both the first and last times it was triggered.
  • Request details, including the URL triggering the event, the PHP version, and which ZendPHP server triggered it.
  • PHP superglobal values (as configured by the zendhq.monitor.report_super_globals ZendPHP extension setting).

Additionally, depending on the event type, additional details may be present, such as a PHP error type, an exception stack trace, or custom details as provided to the monitor_custom_event()/monitor_custom_event_ex() functions.

If you are no longer interested in seeing this event, you may select Delete in the bottom left of the callout; otherwise, select the < in the upper left to close it and return to the event log.

Deleting multiple events from the log

To delete multiple events from the event log, select each you wish to delete via the check box on the left side of the row for each event. Finally, select Delete selected in the upper left of the Event Log screen.

Searching events

The Search box allows you to search by rule name. Note that search terms are case sensitive.