youtubebeat/vendor/github.com/elastic/beats/libbeat/docs/shared-template-load.asciidoc

279 lines
8.8 KiB
Text
Raw Normal View History

2018-11-18 11:08:38 +01:00
//////////////////////////////////////////////////////////////////////////
//// This content is shared by all Elastic Beats. Make sure you keep the
//// descriptions here generic enough to work for all Beats that include
//// this file. When using cross references, make sure that the cross
//// references resolve correctly for any files that include this one.
//// Use the appropriate variables defined in the index.asciidoc file to
//// resolve Beat names: beatname_uc and beatname_lc
//// Use the following include to pull this content into a doc file:
//// include::../../libbeat/docs/shared-template-load.asciidoc[]
//// If you want to include conditional content, you also need to
//// add the following doc attribute definition before the
//// include statement so that you have:
//// :allplatforms:
//// include::../../libbeat/docs/shared-template-load.asciidoc[]
//// This content must be embedded underneath a level 3 heading.
//////////////////////////////////////////////////////////////////////////
In Elasticsearch, {elasticsearch}/indices-templates.html[index
templates] are used to define settings and mappings that determine how fields
should be analyzed.
The recommended index template file for {beatname_uc} is installed by the
{beatname_uc} packages. If you accept the default configuration in the
+{beatname_lc}.yml+ config file, {beatname_uc} loads the template automatically
after successfully connecting to Elasticsearch. If the template already exists,
it's not overwritten unless you configure {beatname_uc} to do so.
You can disable automatic template loading, or load your own template, by
configuring template loading options in the {beatname_uc} configuration file.
You can also set options to change the name of the index and index template.
ifndef::only-elasticsearch[]
NOTE: A connection to Elasticsearch is required to load the index template. If
the output is not Elasticsearch, you must
<<load-template-manually,load the template manually>>.
endif::[]
For more information, see:
ifdef::only-elasticsearch[]
* <<load-template-auto>>
* <<load-template-manually>>
endif::[]
ifndef::only-elasticsearch[]
* <<load-template-auto>>
* <<load-template-manually>> - required for non-Elasticsearch output
endif::[]
[[load-template-auto]]
==== Configure template loading
By default, {beatname_uc} automatically loads the recommended template file,
+fields.yml+, if the Elasticsearch output is enabled. You can change the
defaults in the +{beatname_lc}.yml+ config file to:
* **Load a different template**
+
[source,yaml]
-----
setup.template.name: "your_template_name"
setup.template.fields: "path/to/fields.yml"
-----
+
If the template already exists, its not overwritten unless you configure
{beatname_uc} to do so.
* **Overwrite an existing template**
+
[source,yaml]
-----
setup.template.overwrite: true
-----
* **Disable automatic template loading**
+
[source,yaml]
-----
setup.template.enabled: false
-----
+
If you disable automatic template loading, you need to
<<load-template-manually,load the template manually>>.
* **Change the index name**
+
By default, {beatname_uc} writes events to indices named
+{beatname_lc}-{version}-yyyy.MM.dd+, where `yyyy.MM.dd` is the date when the
events were indexed. To use a different name, you set the
<<index-option-es,`index`>> option in the Elasticsearch output. The value
that you specify should include the root name of the index plus version and
date information. You also need to configure the `setup.template.name` and
`setup.template.pattern` options to match the new name. For example:
+
ifndef::deprecate_dashboard_loading[]
["source","sh",subs="attributes,callouts"]
-----
output.elasticsearch.index: "customname-%{[beat.version]}-%{+yyyy.MM.dd}"
setup.template.name: "customname"
setup.template.pattern: "customname-*"
setup.dashboards.index: "customname-*" <1>
-----
<1> If you plan to
<<load-kibana-dashboards, set up the Kibana dashboards>>, also set
this option to overwrite the index name defined in the dashboards and index
pattern.
endif::[]
ifdef::deprecate_dashboard_loading[]
["source","sh",subs="attributes,callouts"]
-----
output.elasticsearch.index: "customname-%{[beat.version]}-%{+yyyy.MM.dd}"
setup.template.name: "customname"
setup.template.pattern: "customname-*"
-----
+
Also ensure to change the index name accordingly in the Kibana dashboards,
when loading via the Kibana UI.
endif::[]
See <<configuration-template>> for the full list of configuration options.
[[load-template-manually]]
==== Load the template manually
To load the template manually, run the <<setup-command,`setup`>> command. A
connection to Elasticsearch is required.
ifndef::only-elasticsearch[]
If another output is enabled, you need
to temporarily disable that output and enable Elasticsearch by using the
`-E` option. The examples here assume that Logstash output is enabled. You can
omit the `-E` flags if Elasticsearch output is already enabled.
endif::[]
If you are connecting to a secured Elasticsearch cluster, make sure you've
configured credentials as described in <<{beatname_lc}-configuration>>.
If the host running {beatname_uc} does not have direct connectivity to
Elasticsearch, see <<load-template-manually-alternate>>.
To load the template, use the appropriate command for your system.
ifndef::only-elasticsearch[]
:disable_logstash: {sp}-E output.logstash.enabled=false
endif::[]
ifdef::only-elasticsearch[]
:disable_logstash:
endif::[]
ifdef::allplatforms[]
ifeval::["{requires-sudo}"=="yes"]
include::./shared-note-sudo.asciidoc[]
endif::[]
*deb and rpm:*
["source","sh",subs="attributes"]
----
{beatname_lc} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]'
----
*mac:*
["source","sh",subs="attributes"]
----
./{beatname_lc} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]'
----
*docker:*
["source","sh",subs="attributes"]
----------------------------------------------------------------------
docker run {dockerimage} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]'
----------------------------------------------------------------------
*win:*
endif::allplatforms[]
Open a PowerShell prompt as an Administrator (right-click the PowerShell icon
and select *Run As Administrator*).
From the PowerShell prompt, change to the directory where you installed {beatname_uc},
and run:
["source","sh",subs="attributes"]
----------------------------------------------------------------------
PS > .{backslash}{beatname_lc}.exe setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]'
----------------------------------------------------------------------
[[force-kibana-new]]
===== Force Kibana to look at newest documents
If you've already used {beatname_uc} to index data into Elasticsearch,
the index may contain old documents. After you load the index template,
you can delete the old documents from +{beatname_lc}-*+ to force Kibana to look
at the newest documents. Use this command:
*deb, rpm, and mac:*
["source","sh",subs="attributes"]
----------------------------------------------------------------------
curl -XDELETE 'http://localhost:9200/{beatname_lc}-*'
----------------------------------------------------------------------
*win:*
["source","sh",subs="attributes"]
----------------------------------------------------------------------
PS > Invoke-RestMethod -Method Delete "http://localhost:9200/{beatname_lc}-*"
----------------------------------------------------------------------
This command deletes all indices that match the pattern +{beat_default_index_prefix}-*+.
Before running this command, make sure you want to delete all indices that match
the pattern.
[[load-template-manually-alternate]]
==== Load the template manually (alternate method)
If the host running {beatname_uc} does not have direct connectivity to
Elasticsearch, you can export the index template to a file, move it to a
machine that does have connectivity, and then install the template manually.
. Export the index template:
+
ifdef::allplatforms[]
*deb and rpm:*
+
["source","sh",subs="attributes"]
----
{beatname_lc} export template > {beatname_lc}.template.json
----
+
*mac:*
+
["source","sh",subs="attributes"]
----
./{beatname_lc} export template > {beatname_lc}.template.json
----
+
*win:*
+
endif::allplatforms[]
["source","sh",subs="attributes"]
----
PS > .{backslash}{beatname_lc}.exe export template --es.version {stack-version} | Out-File -Encoding UTF8 {beatname_lc}.template.json
----
. Install the template:
+
*deb, rpm, and mac:*
+
["source","sh",subs="attributes"]
----
curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_template/{beatname_lc}-{stack-version} -d@{beatname_lc}.template.json
----
+
*win:*
+
["source","sh",subs="attributes"]
----
PS > Invoke-RestMethod -Method Put -ContentType "application/json" -InFile {beatname_lc}.template.json -Uri http://localhost:9200/_template/{beatname_lc}-{stack-version}
----