////////////////////////////////////////////////////////////////////////// //// This content is shared by all Elastic Beats. Make sure you keep the //// descriptions here generic enough to work for all Beats that include //// this file. When using cross references, make sure that the cross //// references resolve correctly for any files that include this one. //// Use the appropriate variables defined in the index.asciidoc file to //// resolve Beat names: beatname_uc and beatname_lc //// Use the following include to pull this content into a doc file: //// include::../../libbeat/docs/shared-template-load.asciidoc[] //// If you want to include conditional content, you also need to //// add the following doc attribute definition before the //// include statement so that you have: //// :allplatforms: //// include::../../libbeat/docs/shared-template-load.asciidoc[] //// This content must be embedded underneath a level 3 heading. ////////////////////////////////////////////////////////////////////////// In Elasticsearch, {elasticsearch}/indices-templates.html[index templates] are used to define settings and mappings that determine how fields should be analyzed. The recommended index template file for {beatname_uc} is installed by the {beatname_uc} packages. If you accept the default configuration in the +{beatname_lc}.yml+ config file, {beatname_uc} loads the template automatically after successfully connecting to Elasticsearch. If the template already exists, it's not overwritten unless you configure {beatname_uc} to do so. You can disable automatic template loading, or load your own template, by configuring template loading options in the {beatname_uc} configuration file. You can also set options to change the name of the index and index template. ifndef::only-elasticsearch[] NOTE: A connection to Elasticsearch is required to load the index template. If the output is not Elasticsearch, you must <>. endif::[] For more information, see: ifdef::only-elasticsearch[] * <> * <> endif::[] ifndef::only-elasticsearch[] * <> * <> - required for non-Elasticsearch output endif::[] [[load-template-auto]] ==== Configure template loading By default, {beatname_uc} automatically loads the recommended template file, +fields.yml+, if the Elasticsearch output is enabled. You can change the defaults in the +{beatname_lc}.yml+ config file to: * **Load a different template** + [source,yaml] ----- setup.template.name: "your_template_name" setup.template.fields: "path/to/fields.yml" ----- + If the template already exists, it’s not overwritten unless you configure {beatname_uc} to do so. * **Overwrite an existing template** + [source,yaml] ----- setup.template.overwrite: true ----- * **Disable automatic template loading** + [source,yaml] ----- setup.template.enabled: false ----- + If you disable automatic template loading, you need to <>. * **Change the index name** + By default, {beatname_uc} writes events to indices named +{beatname_lc}-{version}-yyyy.MM.dd+, where `yyyy.MM.dd` is the date when the events were indexed. To use a different name, you set the <> option in the Elasticsearch output. The value that you specify should include the root name of the index plus version and date information. You also need to configure the `setup.template.name` and `setup.template.pattern` options to match the new name. For example: + ifndef::deprecate_dashboard_loading[] ["source","sh",subs="attributes,callouts"] ----- output.elasticsearch.index: "customname-%{[beat.version]}-%{+yyyy.MM.dd}" setup.template.name: "customname" setup.template.pattern: "customname-*" setup.dashboards.index: "customname-*" <1> ----- <1> If you plan to <>, also set this option to overwrite the index name defined in the dashboards and index pattern. endif::[] ifdef::deprecate_dashboard_loading[] ["source","sh",subs="attributes,callouts"] ----- output.elasticsearch.index: "customname-%{[beat.version]}-%{+yyyy.MM.dd}" setup.template.name: "customname" setup.template.pattern: "customname-*" ----- + Also ensure to change the index name accordingly in the Kibana dashboards, when loading via the Kibana UI. endif::[] See <> for the full list of configuration options. [[load-template-manually]] ==== Load the template manually To load the template manually, run the <> command. A connection to Elasticsearch is required. ifndef::only-elasticsearch[] If another output is enabled, you need to temporarily disable that output and enable Elasticsearch by using the `-E` option. The examples here assume that Logstash output is enabled. You can omit the `-E` flags if Elasticsearch output is already enabled. endif::[] If you are connecting to a secured Elasticsearch cluster, make sure you've configured credentials as described in <<{beatname_lc}-configuration>>. If the host running {beatname_uc} does not have direct connectivity to Elasticsearch, see <>. To load the template, use the appropriate command for your system. ifndef::only-elasticsearch[] :disable_logstash: {sp}-E output.logstash.enabled=false endif::[] ifdef::only-elasticsearch[] :disable_logstash: endif::[] ifdef::allplatforms[] ifeval::["{requires-sudo}"=="yes"] include::./shared-note-sudo.asciidoc[] endif::[] *deb and rpm:* ["source","sh",subs="attributes"] ---- {beatname_lc} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]' ---- *mac:* ["source","sh",subs="attributes"] ---- ./{beatname_lc} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]' ---- *docker:* ["source","sh",subs="attributes"] ---------------------------------------------------------------------- docker run {dockerimage} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]' ---------------------------------------------------------------------- *win:* endif::allplatforms[] Open a PowerShell prompt as an Administrator (right-click the PowerShell icon and select *Run As Administrator*). From the PowerShell prompt, change to the directory where you installed {beatname_uc}, and run: ["source","sh",subs="attributes"] ---------------------------------------------------------------------- PS > .{backslash}{beatname_lc}.exe setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]' ---------------------------------------------------------------------- [[force-kibana-new]] ===== Force Kibana to look at newest documents If you've already used {beatname_uc} to index data into Elasticsearch, the index may contain old documents. After you load the index template, you can delete the old documents from +{beatname_lc}-*+ to force Kibana to look at the newest documents. Use this command: *deb, rpm, and mac:* ["source","sh",subs="attributes"] ---------------------------------------------------------------------- curl -XDELETE 'http://localhost:9200/{beatname_lc}-*' ---------------------------------------------------------------------- *win:* ["source","sh",subs="attributes"] ---------------------------------------------------------------------- PS > Invoke-RestMethod -Method Delete "http://localhost:9200/{beatname_lc}-*" ---------------------------------------------------------------------- This command deletes all indices that match the pattern +{beat_default_index_prefix}-*+. Before running this command, make sure you want to delete all indices that match the pattern. [[load-template-manually-alternate]] ==== Load the template manually (alternate method) If the host running {beatname_uc} does not have direct connectivity to Elasticsearch, you can export the index template to a file, move it to a machine that does have connectivity, and then install the template manually. . Export the index template: + ifdef::allplatforms[] *deb and rpm:* + ["source","sh",subs="attributes"] ---- {beatname_lc} export template > {beatname_lc}.template.json ---- + *mac:* + ["source","sh",subs="attributes"] ---- ./{beatname_lc} export template > {beatname_lc}.template.json ---- + *win:* + endif::allplatforms[] ["source","sh",subs="attributes"] ---- PS > .{backslash}{beatname_lc}.exe export template --es.version {stack-version} | Out-File -Encoding UTF8 {beatname_lc}.template.json ---- . Install the template: + *deb, rpm, and mac:* + ["source","sh",subs="attributes"] ---- curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_template/{beatname_lc}-{stack-version} -d@{beatname_lc}.template.json ---- + *win:* + ["source","sh",subs="attributes"] ---- PS > Invoke-RestMethod -Method Put -ContentType "application/json" -InFile {beatname_lc}.template.json -Uri http://localhost:9200/_template/{beatname_lc}-{stack-version} ----