295 lines
10 KiB
Plaintext
295 lines
10 KiB
Plaintext
[[configuration-autodiscover]]
|
|
== Autodiscover
|
|
|
|
When you run applications on containers, they become moving targets to the monitoring system. Autodiscover
|
|
allows you to track them and adapt settings as changes happen. By defining configuration templates, the
|
|
autodiscover subsystem can monitor services as they start running.
|
|
|
|
You define autodiscover settings in the +{beatname_lc}.autodiscover+ section of the +{beatname_lc}.yml+
|
|
config file. To enable autodiscover, you specify a list of providers.
|
|
|
|
[float]
|
|
=== Providers
|
|
|
|
Autodiscover providers work by watching for events on the system and translating those events into internal autodiscover
|
|
events with a common format. When you configure the provider, you can use fields from the autodiscover event to set
|
|
conditions that, when met, launch specific configurations.
|
|
|
|
On start, {beatname_uc} will scan existing containers and launch the proper configs for them. Then it will watch for new
|
|
start/stop events. This ensures you don't need to worry about state, but only define your desired configs.
|
|
|
|
[float]
|
|
===== Docker
|
|
|
|
The Docker autodiscover provider watches for Docker containers to start and stop. These are the available fields on every
|
|
event:
|
|
|
|
* host
|
|
* port
|
|
* docker.container.id
|
|
* docker.container.image
|
|
* docker.container.name
|
|
* docker.container.labels
|
|
|
|
|
|
For example:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------------
|
|
{
|
|
"host": "10.4.15.9",
|
|
"port": 6379,
|
|
"docker": {
|
|
"container": {
|
|
"id": "382184ecdb385cfd5d1f1a65f78911054c8511ae009635300ac28b4fc357ce51"
|
|
"name": "redis",
|
|
"image": "redis:3.2.11",
|
|
"labels": {
|
|
"io.kubernetes.pod.namespace": "default"
|
|
...
|
|
}
|
|
}
|
|
}
|
|
}
|
|
-------------------------------------------------------------------------------------
|
|
|
|
You can define a set of configuration templates to be applied when the condition matches an event. Templates define
|
|
a condition to match on autodiscover events, together with the list of configurations to launch when this condition
|
|
happens.
|
|
|
|
Conditions match events from the provider. Providers use the same format for <<conditions>> that
|
|
processors use.
|
|
|
|
Configuration templates can contain variables from the autodiscover event. They can be accessed under the `data` namespace.
|
|
For example, with the example event, "`${data.port}`" resolves to `6379`.
|
|
|
|
include::../../{beatname_lc}/docs/autodiscover-docker-config.asciidoc[]
|
|
|
|
|
|
ifeval::["{beatname_lc}"=="filebeat"]
|
|
[WARNING]
|
|
=======================================
|
|
When using autodiscover, you have to be careful when defining config templates, especially if they are
|
|
reading from places holding information for several containers. For instance, under this file structure:
|
|
|
|
`/mnt/logs/<container_id>/*.log`
|
|
|
|
You can define a config template like this:
|
|
|
|
**Wrong settings**:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------------
|
|
autodiscover.providers:
|
|
- type: docker
|
|
templates:
|
|
- condition.contains:
|
|
docker.container.image: nginx
|
|
config:
|
|
- type: log
|
|
paths:
|
|
- "/mnt/logs/*/*.log"
|
|
-------------------------------------------------------------------------------------
|
|
|
|
That would read all the files under the given path several times (one per nginx container). What you really
|
|
want is to scope your template to the container that matched the autodiscover condition. Good settings:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------------
|
|
autodiscover.providers:
|
|
- type: docker
|
|
templates:
|
|
- condition.contains:
|
|
docker.container.image: nginx
|
|
config:
|
|
- type: log
|
|
paths:
|
|
- "/mnt/logs/${data.docker.container.id}/*.log"
|
|
-------------------------------------------------------------------------------------
|
|
|
|
=======================================
|
|
endif::[]
|
|
|
|
[float]
|
|
===== Kubernetes
|
|
|
|
The Kubernetes autodiscover provider watches for Kubernetes pods to start, update, and stop. These are the available fields on every
|
|
event:
|
|
|
|
* host
|
|
* port (if exposed)
|
|
* kubernetes.container.id
|
|
* kubernetes.container.image
|
|
* kubernetes.container.name
|
|
* kubernetes.labels
|
|
* kubernetes.namespace
|
|
* kubernetes.node.name
|
|
* kubernetes.pod.name
|
|
|
|
If the `include_annotations` config is added to the provider config, then the list of annotations present in the config
|
|
are added to the event.
|
|
|
|
For example:
|
|
|
|
[source,yaml]
|
|
-------------------------------------------------------------------------------------
|
|
{
|
|
"host": "172.17.0.21",
|
|
"port": 9090,
|
|
"kubernetes": {
|
|
"container": {
|
|
"id": "bb3a50625c01b16a88aa224779c39262a9ad14264c3034669a50cd9a90af1527",
|
|
"image": "prom/prometheus",
|
|
"name": "prometheus"
|
|
},
|
|
"labels": {
|
|
"project": "prometheus",
|
|
...
|
|
},
|
|
"namespace": "default",
|
|
"node": {
|
|
"name": "minikube"
|
|
},
|
|
"pod": {
|
|
"name": "prometheus-2657348378-k1pnh"
|
|
}
|
|
},
|
|
}
|
|
-------------------------------------------------------------------------------------
|
|
|
|
The configuration of templates and conditions is similar to that of the Docker provider. Configuration templates can
|
|
contain variables from the autodiscover event. They can be accessed under data namespace.
|
|
|
|
The `kubernetes` autodiscover provider has the following configuration settings:
|
|
|
|
`in_cluster`:: (Optional) Use in cluster settings for Kubernetes client, `true`
|
|
by default.
|
|
`host`:: (Optional) Identify the node where {beatname_lc} is running in case it
|
|
cannot be accurately detected, as when running {beatname_lc} in host network
|
|
mode.
|
|
`kube_config`:: (Optional) Use given config file as configuration for Kubernetes
|
|
client.
|
|
|
|
include::../../{beatname_lc}/docs/autodiscover-kubernetes-config.asciidoc[]
|
|
|
|
[float]
|
|
===== Manually Defining Ports with Kubernetes
|
|
|
|
Declare exposed ports in your pod spec if possible. Otherwise, you will need to use
|
|
multiple templates with complex filtering rules. The {port} variable will not be
|
|
present, and you will need to hardcode ports. Example: `{data.host}:1234`
|
|
|
|
When ports are not declared, Autodiscover generates a config using your provided
|
|
template once per pod, and once per container. These generated configs are
|
|
de-duplicated after they are generated. If the generated configs for multiple
|
|
containers are identical, they will be merged into one config.
|
|
|
|
Pods share an identical host. If only the `{data.host}` variable is interpolated,
|
|
then one config will be generated per host. The configs will be identical.
|
|
After they are de-duplicated, only one will be used.
|
|
|
|
ifdef::autodiscoverJolokia[]
|
|
[float]
|
|
===== Jolokia (experimental)
|
|
|
|
The Jolokia autodiscover provider uses Jolokia Discovery to find agents running
|
|
in your host or your network.
|
|
|
|
Jolokia Discovery mechanism is supported by any Jolokia agent since version
|
|
1.2.0, it is enabled by default when Jolokia is included in the application as
|
|
a JVM agent, but disabled in other cases as the OSGI or WAR (Java EE) agents.
|
|
In any case, this feature is controlled with two properties:
|
|
|
|
* `discoveryEnabled`, to enable the feature
|
|
* `discoveryAgentUrl`, if set, this is the URL announced by the agent when
|
|
being discovered, setting this parameter implicitly enables the feature
|
|
|
|
There are multiple ways of setting these properties, and they can vary from
|
|
application to application, please refer to the documentation of your
|
|
application to find the more suitable way to set them in your case.
|
|
|
|
Jolokia Discovery is based on UDP multicast requests. Agents join the multicast
|
|
group 239.192.48.84, port 24884, and discovery is done by sending queries to
|
|
this group. You have to take into account that UDP traffic between Metricbeat
|
|
and the Jolokia agents has to be allowed. Also notice that this multicast
|
|
address is in the 239.0.0.0/8 range, that is reserved for private use within an
|
|
organization, so it can only be used in private networks.
|
|
|
|
These are the available fields on every event:
|
|
|
|
* jolokia.agent.id
|
|
* jolokia.agent.version
|
|
* jolokia.secured
|
|
* jolokia.server.product
|
|
* jolokia.server.vendor
|
|
* jolokia.server.version
|
|
* jolokia.url
|
|
|
|
The configuration of this provider consists in a set of network interfaces, as
|
|
well as a set of templates as in other providers. The network interfaces will be
|
|
the ones used for discovery probes, they have these settings:
|
|
|
|
`name`:: the name of the interface (e.g. `br0`), it can contain a wildcard
|
|
as suffix to apply the same settings to multiple network interfaces of
|
|
the same type (e.g. `br*`).
|
|
`interval`:: time between probes (defaults to 10s)
|
|
`grace_period`:: time since the last reply to consider an instance stopped
|
|
(defaults to 30s)
|
|
`probe_timeout`:: max time to wait for responses since a probe is sent
|
|
(defaults to 1s)
|
|
|
|
include::../../{beatname_lc}/docs/autodiscover-jolokia-config.asciidoc[]
|
|
endif::autodiscoverJolokia[]
|
|
|
|
ifdef::autodiscoverHints[]
|
|
[[configuration-autodiscover-hints]]
|
|
=== Hints based autodiscover
|
|
|
|
include::../../{beatname_lc}/docs/autodiscover-hints.asciidoc[]
|
|
endif::autodiscoverHints[]
|
|
|
|
[[configuration-autodiscover-advanced]]
|
|
=== Advanced usage
|
|
|
|
////
|
|
Builders are not exposed in docs, as we only have `hints` builder by the moment,
|
|
and that's covered in previous section
|
|
|
|
[float]
|
|
==== Builders
|
|
Builders allow users to pass hints to autodiscover for it to be able to make decisions
|
|
on how and what kind of configuration should be generated. Each Beat can define its own
|
|
Builders that it can use. Hints are generated based on all information that is passed to
|
|
the provider using a prefix "co.elastic.*". The Kubernetes provider uses annotations and
|
|
the Docker provider uses labels to achieve the same.
|
|
|
|
////
|
|
|
|
[float]
|
|
==== Appenders
|
|
Appenders allow users to append configuration that is already built with the help of either templates
|
|
or builders. Appenders can be configured to be applied only when a required condition is matched. The kind
|
|
of configuration that is applied is specific to each appender.
|
|
|
|
[float]
|
|
===== Config
|
|
The config appender can apply a config on top of the config that was generated by
|
|
templates or builders. The config is applied whenever a provided condition is matched. It is always
|
|
applied if there is no condition provided.
|
|
|
|
["source","yaml",subs="attributes"]
|
|
-------------------------------------------------------------------------------------
|
|
metricbeat.autodiscover:
|
|
providers:
|
|
- type: kubernetes
|
|
templates:
|
|
...
|
|
appenders:
|
|
- type: config
|
|
condition.equals:
|
|
kubernetes.namespace: "prometheus"
|
|
config:
|
|
fields:
|
|
type: monitoring
|
|
-------------------------------------------------------------------------------------
|