Monitoring Rules JSON Structure
ZendHQ monitoring rules define conditions that trigger monitoring events and optional actions that will be executed when a monitoring event is triggered.
ZendHQ monitoring rules are a JSON document with an array of zero to many monitoring rules.
If defined in /opt/zend/zendphp/etc/default_monitor_rules.json
prior to initialization, that document will define the default rules for the ZendHQ instance. Otherwise, you may provide an array of rules as the configuration "value" when setting monitoring rules via the websocket API as follows:
{"id":1,"method":"conf.set","params":{"ns":"monitor","key":"rules","value":[...]}}
Monitoring rule
A monitoring rule must contain the following members:
Member | Description |
---|---|
rule_id | A positive integer value uniquely identifying the rule |
rule_type | A string with the rule type |
rule_name | A string with the human-readable rule name |
conditions | An array with rule conditions |
A monitoring rule can contain the following optional members:
Member | Description |
---|---|
rule_description | A string with the human-readable description of the rule |
actions | An array of actions |
Monitoring rules of the types function-slow-exec
and function-error
must contain the following additional members:
Member | Description |
---|---|
regex_match | A regular expression to watch specific PHP functions |
rule_id
The rule_id member is a positive integer value (zero included) uniquely identifying the rule.
rule_type
The rule_type member specifies the type of the rule and must be one of the following:
Value | Description |
---|---|
function-slow-exec | A PHP function took too long time to execute |
function-error | A PHP function returned false |
request-slow-exec | A request took too long time to execute |
request-high-mem-usage | A request used too much memory |
php-error | A PHP error occurred |
custom | A custom event was generated in the PHP script |
-
A
function-slow-exec
rule is triggered when a watched function takes longer to execute than a specified threshold. -
A
function-error
rule is triggered when a watched function returnsfalse
; these are generally used to monitor PHP internal functions that use a return value offalse
to indicate an error. -
A
request-slow-exec
rule is triggered when a request takes longer to execute than a specified threshold. -
A
request-high-mem-usage
rule is triggered when more memory than a specified threshold is in use at the end of the request. -
A
php-error
rule is triggered when a PHP error matching a given PHP error mask occurs. -
A
custom
rule is triggered when the event is generated in the PHP script. See Custom Monitoring Events for details how custom events are generated in the PHP script.
rule_name
The rule_name member is a string uniquely identifying the rule.
conditions
The conditions member is an array of rule conditions. The array must contain at least one condition.
rule_description
The rule_description is a human-readable string describing the rule.
actions
The actions member is an array of actions.
regex_match
The regex_match member is a regular expression used to match the rule against specific PHP internal or user functions. The rule is applied only if the name of the PHP function or method name matches the regular expression. Specify method names in the format {Class Name}::{Method Name}
(for example, PDO::connect
).
See https://github.com/google/re2/wiki/Syntax for syntax.
Monitoring rule conditions
Monitoring rule conditions specify the criteria for monitoring events. Monitoring rule conditions depend on the type of the rule, but must always contain the following members:
Member | Description |
---|---|
condition_id | A positive integer value uniquely identifying the condition |
severity | One of "critical", "warning", or "notice" |
Monitoring rule conditions for rules of the types function-slow-exec
, request-slow-exec
, and request-high-mem-usage
must contain the following additional member:
Member | Description |
---|---|
threshold | A positive number value |
Monitoring rule conditions for the php-error
type must contain the following additional member:
Member | Description |
---|---|
mask | PHP error mask |
condition_id
The condition_id
member is a positive integer value (zero included) uniquely identifying the condition within the rule.
severity
The severity
member specifies the severity of the monitoring event generated when the condition is met.
threshold
The threshold
member specifies the threshold at which the monitoring event with the given severity is generated.
For monitoring rules of the types function-slow-exec
and request-slow-exec
, the threshold value is a time in milliseconds.
For the request-high-mem-usage
monitoring rule, the threshold value is a number of bytes in use at the end of the request.
mask
The mask member specifies a bitmask with PHP error types that generate the monitoring event with the given severity.
The mask can have the following values combined with a bitwise OR operation:
Value | PHP error type |
---|---|
0x0001 | ERROR |
0x0002 | WARNING |
0x0004 | PARSE ERROR |
0x0008 | NOTICE |
0x0010 | CORE ERROR |
0x0020 | CORE WARNING |
0x0040 | COMPILE ERROR |
0x0080 | COMPILE WARNING |
0x0100 | USER ERROR |
0x0200 | USER WARNING |
0x0400 | USER NOTICE |
0x0800 | STRICT |
0x1000 | RECOVERABLE ERROR |
0x2000 | DEPRECATED |
0x4000 | USER DEPRECATED |
Monitoring rule actions
Monitoring rule actions specify actions that are executed when a monitoring event is generated.
Monitoring rule actions must contain the following members:
Member | Description |
---|---|
action_id | A positive integer value uniquely identifying the action |
action_type | One of "url" or "email" |
action_target | A string specifying the target of the action |
action_id
The action_id is a positive integer value (zero included) uniquely identifying the action within the monitoring rule.
action_type
The action_type member specifies the type of the action and must have one of the following values:
Value | Description |
---|---|
url | A webhook that receives notifications for monitoring events |
codetrace | Triggers the code trace collection and storage |
The url action performs an HTTP POST request with the monitoring event data to the given URL. See ZendHQ Monitoring Event Data for monitoring event data details.
action_target
The action_target member specifies the target of the action.
For "url" actions this is the url for the HTTP POST request.
ZendHQ Monitoring Event Data
If a monitoring rule contains monitoring event actions and the event is generated, then a JSON document with monitoring event data is included in the action.
The JSON document contains the following members:
Member | Description |
---|---|
event_id | A number uniquely identifying the event |
issue_id | A number uniquely identifying the issue |
request_id | A number uniquely identifying the request |
name | Name of the monitoring rule |
type | Type of the monitoring rule |
severity | One of "critical", "warning", or "notice" |
time_sec | UNIX time of the event as seconds since the EPOCH |
request | JSON object with request details |
function | Optional JSON object with PHP function details |
error | Optional JSON object with PHP error details |
stack_trace | Optional JSON array with stack trace frames |
custom | Optional JSON object with custom event details |
variables | Optional JSON object with server variables |
duration_sec | Optional number of seconds |
memory_usage_bytes | Optional number of bytes |
event_id
The event_id member contains the ID of the monitoring event.
issue_id
The issue_id member contains the ID of the monitoring issue.
If event aggregation is enabled, then multiple monitoring events can have the same issue ID value.
request_id
The request_id member contains the ID of the request that caused the monitoring event.
If multiple monitoring events are generated during a request, they all have the same request_id value.
name
The name member contains the name of the monitoring rule (see rule_name) that triggered the event.
type
The type member contains the type of the monitoring rule (see rule_type) that triggered the event.
severity
The severity member is the severity of the monitoring event.
time_sec
The time_sec member contains the UNIX time of the event as a number of seconds since the EPOCH (January 1, 1970 midnight UTC/GMT).
request
The request member contains request details and has the following members:
Member | Description |
---|---|
url | The requested target url |
php_version | The PHP version string, for example, "8.0.19" |
node_name | Name of the PHP node (uname or custom) |
pid | PID of the PHP process that processed the request |
function
The optional function member contains PHP function details and has the following members:
Member | Description |
---|---|
function_name | Name of the PHP function |
function_args | Optional JSON array with function argument values |
file_name | Optional PHP script file name |
line_no | Optional line number in the PHP script file |
error
The optional error member contains PHP error details and has the following members:
Member | Description |
---|---|
message | The PHP error message text |
error_type | The PHP error type as a numeric value |
error_type_str | The PHP error type as a text |
file_name | PHP script file name where the error occurred |
line_no | Line number in the PHP script file where the error occurred |
stack_trace
The optional stack_trace member contains a JSON array with the PHP stack trace frames. Stack trace frames in the array contain the following members:
Member | Description |
---|---|
function_name | Name of the PHP function |
file_name | Optional PHP script file name |
line_no | Optional line number in the PHP script file |
custom
The optional custom member contains custom event details and has the following members:
Member | Description |
---|---|
type | A type string |
text | A text |
user_data | User data value |
See Custom Monitoring Events for details.
variables
The optional variables member contains server variables and their values, such as the content of the $_SERVER
variable. See PHP values for the details how PHP values are stored in JSON documents.
duration_sec
The optional duration_sec member contains the duration of a function or a request execution that triggered the event as a number of seconds.
memory_usage_bytes
The optional memory_usage_bytes member contains the number of used memory that triggered the event as a number of bytes.
Sample monitoring event data
{ "event_id": 95, "issue_id": 95, "memory_usage_bytes": 49166992, "name": "High Memory Usage", "request": { "node_name": "node1", "php_version": "8.1.5", "pid": 68381, "url": "http://example.com/monitor.php?high_mem_usage" }, "request_id": 321, "severity": "warning", "time_sec": 1652341693.624875, "type": "request-high-mem-usage", "variables": { "_REQUEST": { "high_mem_usage": "" }, "_SERVER": { "DOCUMENT_ROOT": "/srv/http", "HTTP_ACCEPT": "text/html,application/xhtml xml,application/xml;q=0.9,*/*;q=0.8", "HTTP_ACCEPT_ENCODING": "gzip, deflate", "HTTP_ACCEPT_LANGUAGE": "en-GB,en;q=0.9", "HTTP_CONNECTION": "keep-alive", "HTTP_HOST": "example.com", "HTTP_REFERER": "http://example.com/monitor.php", "HTTP_UPGRADE_INSECURE_REQUESTS": "1", "HTTP_USER_AGENT": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit /605.1.15 (KHTML, like Gecko) Version/15.4 Safari/605.1.15", "PHP_SELF": "/monitor.php", "QUERY_STRING": "high_mem_usage", "REMOTE_ADDR": "127.0.0.1", "REMOTE_PORT": "51914", "REQUEST_METHOD": "GET", "REQUEST_TIME": 1652341693, "REQUEST_TIME_FLOAT": 1652341693.463072, "REQUEST_URI": "/monitor.php?high_mem_usage", "SCRIPT_FILENAME": "/srv/http/monitor.php", "SCRIPT_NAME": "/monitor.php", "SERVER_NAME": "127.0.0.1", "SERVER_PORT": "80", "SERVER_PROTOCOL": "HTTP/1.1", "SERVER_SOFTWARE": "PHP 8.1.5 Development Server" } } }
PHP values
The following rules are applied when PHP values are written into JSON documents:
PHP value | JSON value |
---|---|
string | string |
integer | number |
float | number |
boolean | boolean |
array | array |
object | object with public property values from the PHP object |
any other | null |
Sample Monitoring Rules
Slow function execution rule
This rule triggers when the exec()
PHP internal function or the slow_function()
PHP user function takes too long to execute.
A critical monitoring event is generated if they take 1000 or more milliseconds and a warning monitoring event if they take 500 or more milliseconds to execute.
{ "rule_id": 1, "rule_type": "function-slow-exec", "rule_parent_id": -1, "app_id": -1, "rule_name": "Slow Function Execution", "description": "Triggered when one of the watched functions runs longer than the specified duration", "regex_match": "exec|slow_function", "conditions": [ { "condition_id": 1, "severity": "critical", "threshold": 1000 }, { "condition_id": 2, "severity": "warning", "threshold": 500 } ], "actions": [ { "action_id": 1, "action_type": "url", "action_target": "https://example.com/monitor.php" } ] }
Function error rule
This rule triggers when the fopen()
PHP internal function or the function_that_fails()
PHP user function return false.
The severity of the monitoring event is a warning.
{ "rule_id": 2, "rule_type": "function-error", "rule_parent_id": -1, "app_id": -1, "rule_name": "Function Error", "description": "Triggered when one of the watched functions returns FALSE", "regex_match": "fopen|function_that_fails", "conditions": [ { "condition_id": 1, "severity": "warning" } ], "actions": [ { "action_id": 1, "action_type": "url", "action_target": "https://example.com/monitor.php" } ] }
Slow request execution rule
This rule triggers when the PHP request takes too long to execute.
A critical monitoring event is generated if it takes 5000 or more milliseconds and a warning monitoring event if it takes 2000 or more milliseconds to execute.
{ "rule_id": 3, "rule_type": "request-slow-exec", "rule_parent_id": -1, "app_id": -1, "rule_name": "Slow Request Execution", "description": "Triggered when a PHP request's runtime is longer than the specified duration", "conditions": [ { "condition_id": 1, "severity": "critical", "threshold": 5000 }, { "condition_id": 2, "severity": "warning", "threshold": 2000 } ], "actions": [ { "action_id": 1, "action_type": "url", "action_target": "https://example.com/monitor.php" } ] }
High memory usage rule
This rule triggers when a PHP request uses too much memory.
A critical monitoring event is generated when 49152 or more bytes of memory is in use and a warning monitoring event is generated when 24576 or more bytes of memory is in use at the end of the request.
{ "rule_id": 4, "rule_type": "request-high-mem-usage", "rule_parent_id": -1, "app_id": -1, "rule_name": "High Memory Usage", "description": "Triggered when a PHP request consumes more than the specified amount of memory", "conditions": [ { "condition_id": 1, "severity": "critical", "threshold": 49152 }, { "condition_id": 2, "severity": "warning", "threshold": 24576 } ], "actions": [ { "action_id": 1, "action_type": "url", "action_target": "https://example.com/monitor.php" } ] }
PHP error rule
This rule triggers when one of the PHP errors specified in the conditions occurs.
A critical monitoring event is generated if the PHP error type matches the bitmask 0x0055
(E_ERROR
, E_PARSE_ERROR
, E_CORE_ERROR
, and E_COMPILE_ERROR
) and a warning monitoring event is generated if the PHP error type matches the bitmask 0x17A2
(E_WARNING
, E_CORE_WARNING
, E_COMPILE_WARNING
, E_USER_ERROR
, E_USER_WARNING
, E_USER_NOTICE
, E_RECOVERABLE_ERROR
).
{ "rule_id": 5, "rule_type": "php-error", "rule_parent_id": -1, "app_id": -1, "rule_name": "PHP Error", "description": "Triggered when a PHP error of one of the selected error levels is reported", "conditions": [ { "condition_id": 1, "severity": "critical", "mask": 85 }, { "condition_id": 2, "severity": "warning", "mask": 6050 } ], "actions": [ { "action_id": 1, "action_type": "url", "action_target": "https://example.com/monitor.php" } ] }
Custom event rule
This rule triggers when the monitor_custom_event()
function or the monitor_custom_event_ex($ruleName = "Custom Event")
function are called in the PHP script.
The severity of the monitoring event is a warning unless a specific value is given as the $userSeverity
argument when calling the monitor_custom_event()
function.
{ "rule_id": 6, "rule_type": "custom", "rule_parent_id": -1, "rule_name" : "Custom Event", "description": "Triggered when the API function 'monitor_custom_event' is called", "conditions": [ { "condition_id": 1, "severity": "warning" } ] }