youtubebeat/vendor/github.com/elastic/beats/filebeat/docs/running-on-kubernetes.asciidoc

[[running-on-kubernetes]]
=== Running {beatname_uc} on Kubernetes

You can use {beatname_uc} <<running-on-docker,Docker images>> on Kubernetes to
retrieve and ship container logs.

ifeval::["{release-state}"=="unreleased"]

However, version {stack-version} of {beatname_uc} has not yet been
released, so no Docker image is currently available for this version.

endif::[]


[float]
==== Kubernetes deploy manifests

You deploy {beatname_uc} as a https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/[DaemonSet]
to ensure there's a running instance on each node of the cluster.

The Docker logs host folder (`/var/lib/docker/containers`) is mounted on the
{beatname_uc} container. {beatname_uc} starts an input for the files and
begins harvesting them as soon as they appear in the folder. 

Everything is deployed under the `kube-system` namespace by default. To change
the namespace, modify the manifest file. 

To download the manifest file, run:

["source", "sh", subs="attributes"]
------------------------------------------------
curl -L -O https://raw.githubusercontent.com/elastic/beats/{doc-branch}/deploy/kubernetes/filebeat-kubernetes.yaml
------------------------------------------------

[WARNING]
=======================================
*If you are using Kubernetes 1.7 or earlier:* {beatname_uc} uses a hostPath volume to persist internal data. It's located
under +/var/lib/{beatname_lc}-data+. The manifest uses folder autocreation (`DirectoryOrCreate`), which was introduced in
Kubernetes 1.8. You need to remove `type: DirectoryOrCreate` from the manifest and create the host folder yourself.
=======================================

[float]
==== Settings

By default, {beatname_uc} sends events to an existing Elasticsearch deployment,
if present. To specify a different destination, change the following parameters
in the manifest file:

[source,yaml]
------------------------------------------------
- name: ELASTICSEARCH_HOST
  value: elasticsearch
- name: ELASTICSEARCH_PORT
  value: "9200"
- name: ELASTICSEARCH_USERNAME
  value: elastic
- name: ELASTICSEARCH_PASSWORD
  value: changeme
------------------------------------------------

[float]
===== Red Hat OpenShift configuration

If you are using Red Hat OpenShift, you need to specify additional settings in
the manifest file and enable the container to run as privileged.

. Modify the `DaemonSet` container spec in the manifest file:
+
[source,yaml]
-----
  securityContext:
    runAsUser: 0
    privileged: true
-----

. Grant the `filebeat` service account access to the privileged SCC:
+
[source,shell]
-----
oc adm policy add-scc-to-user privileged system:serviceaccount:kube-system:filebeat
-----
+
This command enables the container to be privileged as an administrator for
OpenShift.

. Override the default node selector for the `kube-system` namespace (or your
custom namespace) to allow for scheduling on any node:
+
[source,shell]
----
oc patch namespace kube-system -p \
'{"metadata": {"annotations": {"openshift.io/node-selector": ""}}}'
----
+
This command sets the node selector for the project to an empty string. If you
don't run this command, the default node selector will skip master nodes.  


[float]
==== Deploy

To deploy {beatname_uc} to Kubernetes, run:

["source", "sh", subs="attributes"]
------------------------------------------------
kubectl create -f filebeat-kubernetes.yaml
------------------------------------------------

To check the status, run:

["source", "sh", subs="attributes"]
------------------------------------------------
$ kubectl --namespace=kube-system get ds/filebeat

NAME       DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE-SELECTOR   AGE
filebeat   32        32        0         32           0           <none>          1m
------------------------------------------------

Log events should start flowing to Elasticsearch. The events are annotated with
metadata added by the <<add-kubernetes-metadata>> processor.