598 lines
20 KiB
Text
598 lines
20 KiB
Text
[[configuration-heartbeat-options]]
|
|
== Set up monitors
|
|
|
|
++++
|
|
<titleabbrev>Set up {beatname_uc} monitors</titleabbrev>
|
|
++++
|
|
|
|
To configure {beatname_uc} define a set of `monitors` to check your remote hosts.
|
|
Specify monitors either directly inside the +heartbeat.yml+ config file, or in external
|
|
dynamically loaded files located in the directory referenced by `heartbeat.config.monitors.path`.
|
|
One advantage of using external files is that these can be automatically reloaded
|
|
without stopping the {beatname_uc} process.
|
|
|
|
Each `monitor` item is an entry in a yaml list, and so begins with a dash (-).
|
|
You can define the type of monitor to use, the hosts to check, and other
|
|
optional settings that control {beatname_uc} behavior.
|
|
|
|
The following example configures three monitors checking via the `icmp`, `tcp`, and `http`
|
|
protocols directly inside the +heartbeat.yml+ file, and demonstrates how to use TCP Echo
|
|
and HTTP response verification:
|
|
|
|
[source,yaml]
|
|
----------------------------------------------------------------------
|
|
# heartbeat.yml
|
|
heartbeat.monitors:
|
|
- type: icmp
|
|
schedule: '*/5 * * * * * *'
|
|
hosts: ["myhost"]
|
|
- type: tcp
|
|
schedule: '@every 5s'
|
|
hosts: ["myhost:7"] # default TCP Echo Protocol
|
|
check.send: "Check"
|
|
check.receive: "Check"
|
|
- type: http
|
|
schedule: '@every 5s'
|
|
urls: ["http://localhost:80/service/status"]
|
|
check.response.status: 200
|
|
heartbeat.scheduler:
|
|
limit: 10
|
|
----------------------------------------------------------------------
|
|
|
|
Using the +heartbeat.yml+ configuration file is convenient, but has two drawbacks:
|
|
it can become hard to manage with large numbers of monitors, and it will not reload
|
|
heartbeat automatically when its contents changes.
|
|
|
|
Define monitors via the +heartbeat.config.monitors+ to prevent those issues from
|
|
happening to you. To do so you would instead have your +heartbeat.yml+ file contain the following:
|
|
|
|
[source,yaml]
|
|
----------------------------------------------------------------------
|
|
# heartbeat.yml
|
|
heartbeat.config.monitors:
|
|
# Directory + glob pattern to search for configuration files
|
|
path: /path/to/my/monitors.d/*.yml
|
|
# If enabled, heartbeat will periodically check the config.monitors path for changes
|
|
reload.enabled: true
|
|
# How often to check for changes
|
|
reload.period: 1s
|
|
----------------------------------------------------------------------
|
|
|
|
Then, define one or more files in the directory pointed to by `heartbeat.config.monitors.path`.
|
|
You may specify multiple monitors in a given file if you like. The contents of these files is
|
|
monitor definitions only, e.g. what is normally under the `heartbeat.monitors` section of
|
|
+heartbeat.yml+. See below for an example
|
|
|
|
[source,yaml]
|
|
----------------------------------------------------------------------
|
|
# /path/to/my/monitors.d/localhost_service_check.yml
|
|
- type: http
|
|
schedule: '@every 5s'
|
|
urls: ["http://localhost:80/service/status"]
|
|
check.response.status: 200
|
|
----------------------------------------------------------------------
|
|
|
|
[float]
|
|
[[monitor-options]]
|
|
=== Monitor options
|
|
|
|
You can specify the following options when defining a {beatname_uc} monitor in any location.
|
|
These options are the same for all monitors. Each monitor type has additional configuration
|
|
options that are specific to that monitor type.
|
|
|
|
[float]
|
|
[[monitor-type]]
|
|
==== `type`
|
|
|
|
The type of monitor to run. One of:
|
|
|
|
* `icmp`: Uses an ICMP (v4 and v6) Echo Request to ping the configured hosts.
|
|
Requires root access. See <<monitor-icmp-options>>.
|
|
* `tcp`: Connects via TCP and optionally verifies the endpoint by sending and/or
|
|
receiving a custom payload. See <<monitor-tcp-options>>.
|
|
* `http`: Connects via HTTP and optionally verifies that the host returns the
|
|
expected response. See <<monitor-http-options>>.
|
|
|
|
The `tcp` and `http` monitor types both support SSL/TLS and some proxy
|
|
settings.
|
|
|
|
|
|
[float]
|
|
[[monitor-name]]
|
|
==== `name`
|
|
|
|
The monitor name. This value appears in the <<exported-fields,exported fields>>
|
|
under the `monitor` field as the job name and the `type` field as the job type.
|
|
|
|
[float]
|
|
[[monitor-enabled]]
|
|
==== `enabled`
|
|
|
|
A Boolean value that specifies whether the module is enabled. If the `enabled`
|
|
option is missing from the configuration block, the module is enabled by
|
|
default.
|
|
|
|
[float]
|
|
[[monitor-schedule]]
|
|
==== `schedule`
|
|
|
|
A cron-like expression that specifies the task schedule. For example:
|
|
|
|
* `*/5 * * * * * *` runs the task every 5 seconds (for example, at 10:00:00,
|
|
10:00:05, and so on).
|
|
* `@every 5s` runs the task every 5 seconds from the time when {beatname_uc} was
|
|
started.
|
|
|
|
The `schedule` option uses a cron-like syntax based on https://github.com/gorhill/cronexpr#implementation[this `cronexpr` implementation],
|
|
but adds the `@every` keyword.
|
|
|
|
[float]
|
|
[[monitor-ipv4]]
|
|
==== `ipv4`
|
|
|
|
A Boolean value that specifies whether to ping using the ipv4 protocol if
|
|
hostnames are configured. The default is `true`.
|
|
|
|
[float]
|
|
[[monitor-ipv6]]
|
|
==== `ipv6`
|
|
|
|
A Boolean value that specifies whether to ping using the ipv6 protocol
|
|
if hostnames are configured. The default is `true`.
|
|
|
|
[float]
|
|
[[monitor-mode]]
|
|
==== `mode`
|
|
|
|
If `mode` is `any`, the monitor pings only one IP address for a hostname. If
|
|
`mode` is `all`, the monitor pings all resolvable IPs for a hostname. The
|
|
`mode: all` setting is useful if you are using a DNS-load balancer and want to
|
|
ping every IP address for the specified hostname. The default is `any`.
|
|
|
|
[float]
|
|
[[monitor-timeout]]
|
|
==== `timeout`
|
|
|
|
The total running time for each ping test. This is the total time allowed for
|
|
testing the connection and exchanging data. The default is 16 seconds (16s).
|
|
|
|
If the timeout is exceeded, {beatname_uc} publishes a `service-down` event. If the
|
|
value specified for `timeout` is greater than `schedule`, intermediate checks
|
|
will not be executed by the scheduler.
|
|
|
|
[float]
|
|
[[monitor-fields]]
|
|
==== `fields`
|
|
|
|
Optional fields that you can specify to add additional information to the
|
|
output. For example, you might add fields that you can use for filtering log
|
|
data. Fields can be scalar values, arrays, dictionaries, or any nested
|
|
combination of these. By default, the fields that you specify here will be
|
|
grouped under a `fields` sub-dictionary in the output document. To store the
|
|
custom fields as top-level fields, set the `fields_under_root` option to true.
|
|
If a duplicate field is declared in the general configuration, then its value
|
|
will be overwritten by the value declared here.
|
|
|
|
[float]
|
|
[[monitor-fields-under-root]]
|
|
==== `fields_under_root`
|
|
|
|
If this option is set to true, the custom <<monitor-fields,fields>>
|
|
are stored as top-level fields in the output document instead of being grouped
|
|
under a `fields` sub-dictionary. If the custom field names conflict with other
|
|
field names added by {beatname_uc}, then the custom fields overwrite the other
|
|
fields.
|
|
|
|
[float]
|
|
[[monitor-tags]]
|
|
==== `tags`
|
|
|
|
A list of tags that will be sent with the monitor event. This setting is optional.
|
|
|
|
[float]
|
|
[[monitor-processors]]
|
|
==== `processors`
|
|
|
|
A list of processors to apply to the data generated by the monitor.
|
|
|
|
See <<filtering-and-enhancing-data>> for information about specifying
|
|
processors in your config.
|
|
|
|
[float]
|
|
[[monitor-icmp-options]]
|
|
=== ICMP options
|
|
|
|
These options configure {beatname_uc} to use ICMP (v4 and v6) Echo Requests to check
|
|
the configured hosts. These options are valid when the <<monitor-type,`type`>> is
|
|
`icmp`.
|
|
|
|
[float]
|
|
[[monitor-icmp-hosts]]
|
|
==== `hosts`
|
|
|
|
A list of hosts to ping.
|
|
|
|
[float]
|
|
[[monitor-icmp-wait]]
|
|
==== `wait`
|
|
|
|
The duration to wait before emitting another ICMP Echo Request. The default is 1
|
|
second (1s).
|
|
|
|
[float]
|
|
[[monitor-tcp-options]]
|
|
=== TCP options
|
|
|
|
These options configure {beatname_uc} to connect via TCP and optionally verify the
|
|
endpoint by sending and/or receiving a custom payload. These options are valid when
|
|
the <<monitor-type,`type`>> is `tcp`.
|
|
|
|
[float]
|
|
[[monitor-tcp-hosts]]
|
|
==== `hosts`
|
|
|
|
A list of hosts to ping. The entries in the list can be:
|
|
|
|
* A plain host name, such as `localhost`, or an IP address. If you specify this
|
|
option, you must also specify a value for <<monitor-tcp-ports,`ports`>>. If the
|
|
monitor is <<configuration-ssl,configured to use SSL>>, {beatname_uc} establishes an
|
|
SSL/TLS-based connection. Otherwise, it establishes a plain TCP connection.
|
|
* A hostname and port, such as `localhost:12345`. {beatname_uc} connects
|
|
to the port on the specified host. If the monitor is
|
|
<<configuration-ssl,configured to use SSL>>, {beatname_uc} establishes an
|
|
SSL/TLS-based connection. Otherwise, it establishes a TCP connection.
|
|
* A full URL using the syntax `scheme://<host>:[port]`, where:
|
|
** `scheme` is one of `tcp`, `plain`, `ssl` or `tls`. If `tcp` or `plain` is
|
|
specified, {beatname_uc} establishes a TCP connection even if the monitor is
|
|
configured to use SSL. If `tls` or `ssl` is specified, {beatname_uc} establishes
|
|
an SSL connection. However, if the monitor is not configured to use SSL, the
|
|
system defaults are used (currently not supported on Windows).
|
|
** `host` is the hostname.
|
|
** `port` is the port number. If `port` is missing in the URL, the
|
|
<<monitor-tcp-ports,`ports`>> setting is required.
|
|
|
|
[float]
|
|
[[monitor-tcp-ports]]
|
|
==== `ports`
|
|
|
|
A list of ports to ping if the host specified in <<monitor-tcp-hosts,`hosts`>>
|
|
does not contain a port number.
|
|
|
|
Example configuration:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------
|
|
- type: tcp
|
|
schedule: '@every 5s'
|
|
hosts: ["myhost"]
|
|
ports: [80, 9200, 5044]
|
|
-------------------------------------------------------------------------------
|
|
|
|
[float]
|
|
[[monitor-tcp-check]]
|
|
==== `check`
|
|
|
|
An optional payload string to send to the remote host and the expected answer.
|
|
If no payload is specified, the endpoint is assumed to be available if the
|
|
connection attempt was successful. If `send` is specified without `receive`,
|
|
any response is accepted as OK. If `receive` is specified without `send`, no
|
|
payload is sent, but the client expects to receive a payload in the form of a
|
|
"hello message" or "banner" on connect.
|
|
|
|
Example configuration:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------
|
|
- type: tcp
|
|
schedule: '@every 5s'
|
|
hosts: ["myhost"]
|
|
ports: [7]
|
|
check.send: 'Hello World'
|
|
check.receive: 'Hello World'
|
|
-------------------------------------------------------------------------------
|
|
|
|
|
|
[float]
|
|
[[monitor-tcp-proxy-url]]
|
|
==== `proxy_url`
|
|
|
|
The URL of the SOCKS5 proxy to use when connecting to the server. The value
|
|
must be a URL with a scheme of socks5://.
|
|
|
|
If the SOCKS5 proxy server requires client authentication, then a username and
|
|
password can be embedded in the URL as shown in the example.
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------
|
|
proxy_url: socks5://user:password@socks5-proxy:2233
|
|
-------------------------------------------------------------------------------
|
|
|
|
When using a proxy, hostnames are resolved on the proxy server instead of on
|
|
the client. You can change this behavior by setting the
|
|
`proxy_use_local_resolver` option.
|
|
|
|
[float]
|
|
[[monitor-tcp-proxy-use-local-resolver]]
|
|
==== `proxy_use_local_resolver`
|
|
|
|
A Boolean value that determines whether hostnames are resolved locally instead
|
|
of being resolved on the proxy server. The default value is false, which means
|
|
that name resolution occurs on the proxy server.
|
|
|
|
[float]
|
|
[[monitor-tcp-tls-ssl]]
|
|
==== `ssl`
|
|
|
|
The TLS/SSL connection settings. If the monitor is
|
|
<<configuration-ssl,configured to use SSL>>, it will attempt an SSL
|
|
handshake. If `check` is not configured, the monitor will only check to see if
|
|
it can establish an SSL/TLS connection. This check can fail either at TCP level
|
|
or during certificate validation.
|
|
|
|
Example configuration:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------
|
|
- type: tcp
|
|
schedule: '@every 5s'
|
|
hosts: ["myhost"]
|
|
ports: [80, 9200, 5044]
|
|
ssl:
|
|
certificate_authorities: ['/etc/ca.crt']
|
|
supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]
|
|
-------------------------------------------------------------------------------
|
|
|
|
|
|
Also see <<configuration-ssl>> for a full description of the `ssl` options.
|
|
|
|
[float]
|
|
[[monitor-http-options]]
|
|
=== HTTP options
|
|
|
|
These options configure {beatname_uc} to connect via HTTP and optionally verify that
|
|
the host returns the expected response. These options are valid when the
|
|
<<monitor-type,`type`>> is `http`.
|
|
|
|
[float]
|
|
[[monitor-http-urls]]
|
|
==== `urls`
|
|
|
|
A list of URLs to ping.
|
|
|
|
Example configuration:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------
|
|
- type: http
|
|
schedule: '@every 5s'
|
|
urls: ["http://myhost:80"]
|
|
-------------------------------------------------------------------------------
|
|
|
|
|
|
[float]
|
|
[[monitor-http-proxy-url]]
|
|
==== `proxy_url`
|
|
|
|
The HTTP proxy URL. This setting is optional.
|
|
|
|
[float]
|
|
[[monitor-http-username]]
|
|
==== `username`
|
|
|
|
The username for authenticating with the server. The credentials are passed
|
|
with the request. This setting is optional.
|
|
|
|
You need to specify credentials when your `check.response` settings require it.
|
|
For example, you can check for a 403 response (`check.response.status: 403`)
|
|
without setting credentials.
|
|
|
|
[float]
|
|
[[monitor-http-password]]
|
|
==== `password`
|
|
|
|
The password for authenticating with the server. This setting is optional.
|
|
|
|
[float]
|
|
[[monitor-http-tls-ssl]]
|
|
==== `ssl`
|
|
|
|
The TLS/SSL connection settings for use with the HTTPS endpoint. If you don't
|
|
specify settings, the system defaults are used.
|
|
|
|
|
|
Example configuration:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------
|
|
- type: http
|
|
schedule: '@every 5s'
|
|
urls: ["https://myhost:443"]
|
|
ssl:
|
|
certificate_authorities: ['/etc/ca.crt']
|
|
supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]
|
|
-------------------------------------------------------------------------------
|
|
|
|
|
|
Also see <<configuration-ssl>> for a full description of the `ssl` options.
|
|
|
|
[float]
|
|
[[monitor-http-check]]
|
|
==== `check`
|
|
|
|
An optional `request` to send to the remote host and the expected `response`.
|
|
|
|
Example configuration:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------
|
|
- type: http
|
|
schedule: '@every 5s'
|
|
urls: ["http://myhost:80"]
|
|
check.request.method: HEAD
|
|
check.response.status: 200
|
|
-------------------------------------------------------------------------------
|
|
|
|
|
|
Under `check.request`, specify these options:
|
|
|
|
*`method`*:: The HTTP method to use. Valid values are `"HEAD"`, `"GET"` and
|
|
`"POST"`.
|
|
*`headers`*:: A dictionary of additional HTTP headers to send.
|
|
*`body`*:: Optional request body content.
|
|
|
|
Under `check.response`, specify these options:
|
|
|
|
*`status`*:: The expected status code. If this setting is not configured or
|
|
it's set to 0, any status code other than 404 is accepted.
|
|
*`headers`*:: The required response headers.
|
|
*`body`*:: A list of regular expressions to match the the body output. Only a single expression needs to match.
|
|
|
|
The following configuration shows how to check the response when the body
|
|
contains JSON:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------
|
|
- type: http
|
|
schedule: '@every 5s'
|
|
urls: ["https://myhost:80"]
|
|
check.request:
|
|
method: GET
|
|
headers:
|
|
'X-API-Key': '12345-mykey-67890'
|
|
check.response:
|
|
status: 200
|
|
body: '{"status": "ok"}'
|
|
-------------------------------------------------------------------------------
|
|
|
|
The following configuration shows how to check the response for multiple regex
|
|
patterns:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------
|
|
- type: http
|
|
schedule: '@every 5s'
|
|
urls: ["https://myhost:80"]
|
|
check.request:
|
|
method: GET
|
|
headers:
|
|
'X-API-Key': '12345-mykey-67890'
|
|
check.response:
|
|
status: 200
|
|
body:
|
|
- hello
|
|
- world
|
|
-------------------------------------------------------------------------------
|
|
|
|
The following configuration shows how to check the response with a multiline
|
|
regex:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------
|
|
- type: http
|
|
schedule: '@every 5s'
|
|
urls: ["https://myhost:80"]
|
|
check.request:
|
|
method: GET
|
|
headers:
|
|
'X-API-Key': '12345-mykey-67890'
|
|
check.response:
|
|
status: 200
|
|
body: '(?s)first.*second.*third'
|
|
-------------------------------------------------------------------------------
|
|
|
|
|
|
[float]
|
|
[[monitors-scheduler]]
|
|
=== Scheduler options
|
|
|
|
You specify options under `heartbeat.scheduler` to control the behavior of the task
|
|
scheduler.
|
|
|
|
Example configuration:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------
|
|
heartbeat.scheduler:
|
|
limit: 10
|
|
location: 'UTC-08:00'
|
|
-------------------------------------------------------------------------------
|
|
|
|
In the example, setting `limit` to 10 guarantees that only 10 concurrent
|
|
I/O tasks will be active. An I/O task can be the actual check or resolving an
|
|
address via DNS.
|
|
|
|
[float]
|
|
[[heartbeat-scheduler-limit]]
|
|
==== `limit`
|
|
|
|
The number of concurrent I/O tasks that {beatname_uc} is allowed to execute. If set
|
|
to 0, there is no limit. The default is 0.
|
|
|
|
Most operating systems set a file descriptor limit of 1024. For {beatname_uc} to
|
|
operate correctly and not accidentally block libbeat output, the value that you
|
|
specify for `limit` should be below the configured ulimit.
|
|
|
|
|
|
[float]
|
|
[[heartbeat-scheduler-location]]
|
|
==== `location`
|
|
|
|
The timezone for the scheduler. By default the scheduler uses localtime.
|
|
|
|
[float]
|
|
[[monitor-watch-poll-file]]
|
|
==== `watch.poll_file`
|
|
|
|
deprecated[6.5.0,Replaced by using dynamic reloading via the
|
|
`heartbeat.config.monitors` option.]
|
|
|
|
The JSON file to watch for additional monitor configurations. The JSON file can
|
|
contain multiple objects, each of which specifies a different monitor config.
|
|
{beatname_uc} checks this file periodically and starts a new monitor instance for
|
|
each new JSON object added to the file. For example, imagine that you add
|
|
10 new entries to the JSON file, each for a different hostname. When {beatname_uc}
|
|
picks up the changes in the file, it merges the original config
|
|
(`heartbeat.yml`) plus the JSON objects, and starts a monitor for each new host
|
|
that you've configured. If you delete an object from the JSON file and it
|
|
doesn't exist in the main config, {beatname_uc} stops the monitor instance running
|
|
for that object.
|
|
|
|
Each monitor has a unique ID that's based on parameters like protocol, host,
|
|
and port. If two monitors have the same ID, {beatname_uc} uses the settings that
|
|
are defined in the last JSON object of the merged config. This means that
|
|
you can specify settings in the JSON file that overwrite the settings in
|
|
the main config. In this way, the configuration that you specify for the
|
|
monitor in the main {beatname_uc} config file acts like a default config that you
|
|
can live-reconfigure by specifying additional configurations in the external
|
|
JSON file.
|
|
|
|
Example configuration:
|
|
|
|
[source, yaml]
|
|
-------------------------------------------------------------------------------
|
|
heartbeat.monitors:
|
|
- type: tcp
|
|
schedule: '*/5 * * * * * *'
|
|
hosts: ["myhost"]
|
|
watch.poll_file:
|
|
path: {path.config}/monitors/dynamic.json
|
|
interval: 5s
|
|
-------------------------------------------------------------------------------
|
|
|
|
*`path`*:: Specifies the path to the JSON file to check for updates.
|
|
*`interval`*:: Specifies how often {beatname_uc} checks the file for changes.
|
|
|
|
To reconfigure the settings specified in the example config, you could define
|
|
the following JSON objects in `dynamic.json`:
|
|
|
|
[source, json]
|
|
-------------------------------------------------------------------------------
|
|
{"hosts": ["myhost:1234"], "schedule": "*/15 * * * * * *"} <1>
|
|
{"hosts": ["tls://otherhost:479"], "ssl.certificate_authorities": ["path/to/ca/file.pem"]} <2>
|
|
-------------------------------------------------------------------------------
|
|
<1> Upon detecting the changes, {beatname_uc} stops the old monitor and then
|
|
restarts it with a schedule of 15 seconds between checks.
|
|
<2> {beatname_uc} starts a new monitor that uses a TLS-based connection with a
|
|
custom CA certificate.
|
|
|
|
|