279 lines
8.8 KiB
Text
279 lines
8.8 KiB
Text
|
//////////////////////////////////////////////////////////////////////////
|
|||
|
|
//// This content is shared by all Elastic Beats. Make sure you keep the
|
|||
|
|
//// descriptions here generic enough to work for all Beats that include
|
|||
|
|
//// this file. When using cross references, make sure that the cross
|
|||
|
|
//// references resolve correctly for any files that include this one.
|
|||
|
|
//// Use the appropriate variables defined in the index.asciidoc file to
|
|||
|
|
//// resolve Beat names: beatname_uc and beatname_lc
|
|||
|
|
//// Use the following include to pull this content into a doc file:
|
|||
|
|
//// include::../../libbeat/docs/shared-template-load.asciidoc[]
|
|||
|
|
//// If you want to include conditional content, you also need to
|
|||
|
|
//// add the following doc attribute definition before the
|
|||
|
|
//// include statement so that you have:
|
|||
|
|
//// :allplatforms:
|
|||
|
|
//// include::../../libbeat/docs/shared-template-load.asciidoc[]
|
|||
|
|
//// This content must be embedded underneath a level 3 heading.
|
|||
|
|
//////////////////////////////////////////////////////////////////////////
|
|||
|
|
|
|||
|
|
|
|||
|
|
In Elasticsearch, {elasticsearch}/indices-templates.html[index
|
|||
|
|
templates] are used to define settings and mappings that determine how fields
|
|||
|
|
should be analyzed.
|
|||
|
|
|
|||
|
|
The recommended index template file for {beatname_uc} is installed by the
|
|||
|
|
{beatname_uc} packages. If you accept the default configuration in the
|
|||
|
|
+{beatname_lc}.yml+ config file, {beatname_uc} loads the template automatically
|
|||
|
|
after successfully connecting to Elasticsearch. If the template already exists,
|
|||
|
|
it's not overwritten unless you configure {beatname_uc} to do so.
|
|||
|
|
|
|||
|
|
You can disable automatic template loading, or load your own template, by
|
|||
|
|
configuring template loading options in the {beatname_uc} configuration file.
|
|||
|
|
|
|||
|
|
You can also set options to change the name of the index and index template.
|
|||
|
|
|
|||
|
|
ifndef::only-elasticsearch[]
|
|||
|
|
NOTE: A connection to Elasticsearch is required to load the index template. If
|
|||
|
|
the output is not Elasticsearch, you must
|
|||
|
|
<<load-template-manually,load the template manually>>.
|
|||
|
|
endif::[]
|
|||
|
|
|
|||
|
|
For more information, see:
|
|||
|
|
|
|||
|
|
ifdef::only-elasticsearch[]
|
|||
|
|
* <<load-template-auto>>
|
|||
|
|
* <<load-template-manually>>
|
|||
|
|
endif::[]
|
|||
|
|
|
|||
|
|
ifndef::only-elasticsearch[]
|
|||
|
|
* <<load-template-auto>>
|
|||
|
|
* <<load-template-manually>> - required for non-Elasticsearch output
|
|||
|
|
endif::[]
|
|||
|
|
|
|||
|
|
[[load-template-auto]]
|
|||
|
|
==== Configure template loading
|
|||
|
|
|
|||
|
|
By default, {beatname_uc} automatically loads the recommended template file,
|
|||
|
|
+fields.yml+, if the Elasticsearch output is enabled. You can change the
|
|||
|
|
defaults in the +{beatname_lc}.yml+ config file to:
|
|||
|
|
|
|||
|
|
* **Load a different template**
|
|||
|
|
+
|
|||
|
|
[source,yaml]
|
|||
|
|
-----
|
|||
|
|
setup.template.name: "your_template_name"
|
|||
|
|
setup.template.fields: "path/to/fields.yml"
|
|||
|
|
-----
|
|||
|
|
+
|
|||
|
|
If the template already exists, it’s not overwritten unless you configure
|
|||
|
|
{beatname_uc} to do so.
|
|||
|
|
|
|||
|
|
* **Overwrite an existing template**
|
|||
|
|
+
|
|||
|
|
[source,yaml]
|
|||
|
|
-----
|
|||
|
|
setup.template.overwrite: true
|
|||
|
|
-----
|
|||
|
|
|
|||
|
|
* **Disable automatic template loading**
|
|||
|
|
+
|
|||
|
|
[source,yaml]
|
|||
|
|
-----
|
|||
|
|
setup.template.enabled: false
|
|||
|
|
-----
|
|||
|
|
+
|
|||
|
|
If you disable automatic template loading, you need to
|
|||
|
|
<<load-template-manually,load the template manually>>.
|
|||
|
|
|
|||
|
|
* **Change the index name**
|
|||
|
|
+
|
|||
|
|
By default, {beatname_uc} writes events to indices named
|
|||
|
|
+{beatname_lc}-{version}-yyyy.MM.dd+, where `yyyy.MM.dd` is the date when the
|
|||
|
|
events were indexed. To use a different name, you set the
|
|||
|
|
<<index-option-es,`index`>> option in the Elasticsearch output. The value
|
|||
|
|
that you specify should include the root name of the index plus version and
|
|||
|
|
date information. You also need to configure the `setup.template.name` and
|
|||
|
|
`setup.template.pattern` options to match the new name. For example:
|
|||
|
|
+
|
|||
|
|
ifndef::deprecate_dashboard_loading[]
|
|||
|
|
|
|||
|
|
["source","sh",subs="attributes,callouts"]
|
|||
|
|
-----
|
|||
|
|
output.elasticsearch.index: "customname-%{[beat.version]}-%{+yyyy.MM.dd}"
|
|||
|
|
setup.template.name: "customname"
|
|||
|
|
setup.template.pattern: "customname-*"
|
|||
|
|
setup.dashboards.index: "customname-*" <1>
|
|||
|
|
-----
|
|||
|
|
|
|||
|
|
<1> If you plan to
|
|||
|
|
<<load-kibana-dashboards, set up the Kibana dashboards>>, also set
|
|||
|
|
this option to overwrite the index name defined in the dashboards and index
|
|||
|
|
pattern.
|
|||
|
|
|
|||
|
|
endif::[]
|
|||
|
|
|
|||
|
|
ifdef::deprecate_dashboard_loading[]
|
|||
|
|
|
|||
|
|
["source","sh",subs="attributes,callouts"]
|
|||
|
|
-----
|
|||
|
|
output.elasticsearch.index: "customname-%{[beat.version]}-%{+yyyy.MM.dd}"
|
|||
|
|
setup.template.name: "customname"
|
|||
|
|
setup.template.pattern: "customname-*"
|
|||
|
|
-----
|
|||
|
|
+
|
|||
|
|
Also ensure to change the index name accordingly in the Kibana dashboards,
|
|||
|
|
when loading via the Kibana UI.
|
|||
|
|
|
|||
|
|
endif::[]
|
|||
|
|
|
|||
|
|
See <<configuration-template>> for the full list of configuration options.
|
|||
|
|
|
|||
|
|
|
|||
|
|
[[load-template-manually]]
|
|||
|
|
==== Load the template manually
|
|||
|
|
|
|||
|
|
To load the template manually, run the <<setup-command,`setup`>> command. A
|
|||
|
|
connection to Elasticsearch is required.
|
|||
|
|
ifndef::only-elasticsearch[]
|
|||
|
|
If another output is enabled, you need
|
|||
|
|
to temporarily disable that output and enable Elasticsearch by using the
|
|||
|
|
`-E` option. The examples here assume that Logstash output is enabled. You can
|
|||
|
|
omit the `-E` flags if Elasticsearch output is already enabled.
|
|||
|
|
endif::[]
|
|||
|
|
|
|||
|
|
If you are connecting to a secured Elasticsearch cluster, make sure you've
|
|||
|
|
configured credentials as described in <<{beatname_lc}-configuration>>.
|
|||
|
|
|
|||
|
|
If the host running {beatname_uc} does not have direct connectivity to
|
|||
|
|
Elasticsearch, see <<load-template-manually-alternate>>.
|
|||
|
|
|
|||
|
|
To load the template, use the appropriate command for your system.
|
|||
|
|
|
|||
|
|
ifndef::only-elasticsearch[]
|
|||
|
|
:disable_logstash: {sp}-E output.logstash.enabled=false
|
|||
|
|
endif::[]
|
|||
|
|
|
|||
|
|
ifdef::only-elasticsearch[]
|
|||
|
|
:disable_logstash:
|
|||
|
|
endif::[]
|
|||
|
|
|
|||
|
|
ifdef::allplatforms[]
|
|||
|
|
|
|||
|
|
ifeval::["{requires-sudo}"=="yes"]
|
|||
|
|
|
|||
|
|
include::./shared-note-sudo.asciidoc[]
|
|||
|
|
|
|||
|
|
endif::[]
|
|||
|
|
|
|||
|
|
*deb and rpm:*
|
|||
|
|
["source","sh",subs="attributes"]
|
|||
|
|
----
|
|||
|
|
{beatname_lc} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]'
|
|||
|
|
----
|
|||
|
|
|
|||
|
|
*mac:*
|
|||
|
|
|
|||
|
|
["source","sh",subs="attributes"]
|
|||
|
|
----
|
|||
|
|
./{beatname_lc} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]'
|
|||
|
|
----
|
|||
|
|
|
|||
|
|
|
|||
|
|
*docker:*
|
|||
|
|
|
|||
|
|
["source","sh",subs="attributes"]
|
|||
|
|
----------------------------------------------------------------------
|
|||
|
|
docker run {dockerimage} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]'
|
|||
|
|
----------------------------------------------------------------------
|
|||
|
|
|
|||
|
|
|
|||
|
|
*win:*
|
|||
|
|
|
|||
|
|
endif::allplatforms[]
|
|||
|
|
|
|||
|
|
Open a PowerShell prompt as an Administrator (right-click the PowerShell icon
|
|||
|
|
and select *Run As Administrator*).
|
|||
|
|
|
|||
|
|
From the PowerShell prompt, change to the directory where you installed {beatname_uc},
|
|||
|
|
and run:
|
|||
|
|
|
|||
|
|
["source","sh",subs="attributes"]
|
|||
|
|
----------------------------------------------------------------------
|
|||
|
|
PS > .{backslash}{beatname_lc}.exe setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]'
|
|||
|
|
----------------------------------------------------------------------
|
|||
|
|
|
|||
|
|
|
|||
|
|
[[force-kibana-new]]
|
|||
|
|
===== Force Kibana to look at newest documents
|
|||
|
|
|
|||
|
|
If you've already used {beatname_uc} to index data into Elasticsearch,
|
|||
|
|
the index may contain old documents. After you load the index template,
|
|||
|
|
you can delete the old documents from +{beatname_lc}-*+ to force Kibana to look
|
|||
|
|
at the newest documents. Use this command:
|
|||
|
|
|
|||
|
|
*deb, rpm, and mac:*
|
|||
|
|
|
|||
|
|
["source","sh",subs="attributes"]
|
|||
|
|
----------------------------------------------------------------------
|
|||
|
|
curl -XDELETE 'http://localhost:9200/{beatname_lc}-*'
|
|||
|
|
----------------------------------------------------------------------
|
|||
|
|
|
|||
|
|
*win:*
|
|||
|
|
|
|||
|
|
["source","sh",subs="attributes"]
|
|||
|
|
----------------------------------------------------------------------
|
|||
|
|
PS > Invoke-RestMethod -Method Delete "http://localhost:9200/{beatname_lc}-*"
|
|||
|
|
----------------------------------------------------------------------
|
|||
|
|
|
|||
|
|
|
|||
|
|
This command deletes all indices that match the pattern +{beat_default_index_prefix}-*+.
|
|||
|
|
Before running this command, make sure you want to delete all indices that match
|
|||
|
|
the pattern.
|
|||
|
|
|
|||
|
|
[[load-template-manually-alternate]]
|
|||
|
|
==== Load the template manually (alternate method)
|
|||
|
|
|
|||
|
|
If the host running {beatname_uc} does not have direct connectivity to
|
|||
|
|
Elasticsearch, you can export the index template to a file, move it to a
|
|||
|
|
machine that does have connectivity, and then install the template manually.
|
|||
|
|
|
|||
|
|
. Export the index template:
|
|||
|
|
+
|
|||
|
|
ifdef::allplatforms[]
|
|||
|
|
*deb and rpm:*
|
|||
|
|
+
|
|||
|
|
["source","sh",subs="attributes"]
|
|||
|
|
----
|
|||
|
|
{beatname_lc} export template > {beatname_lc}.template.json
|
|||
|
|
----
|
|||
|
|
+
|
|||
|
|
*mac:*
|
|||
|
|
+
|
|||
|
|
["source","sh",subs="attributes"]
|
|||
|
|
----
|
|||
|
|
./{beatname_lc} export template > {beatname_lc}.template.json
|
|||
|
|
----
|
|||
|
|
+
|
|||
|
|
*win:*
|
|||
|
|
+
|
|||
|
|
endif::allplatforms[]
|
|||
|
|
["source","sh",subs="attributes"]
|
|||
|
|
----
|
|||
|
|
PS > .{backslash}{beatname_lc}.exe export template --es.version {stack-version} | Out-File -Encoding UTF8 {beatname_lc}.template.json
|
|||
|
|
----
|
|||
|
|
|
|||
|
|
. Install the template:
|
|||
|
|
+
|
|||
|
|
*deb, rpm, and mac:*
|
|||
|
|
+
|
|||
|
|
["source","sh",subs="attributes"]
|
|||
|
|
----
|
|||
|
|
curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_template/{beatname_lc}-{stack-version} -d@{beatname_lc}.template.json
|
|||
|
|
----
|
|||
|
|
+
|
|||
|
|
*win:*
|
|||
|
|
+
|
|||
|
|
["source","sh",subs="attributes"]
|
|||
|
|
----
|
|||
|
|
PS > Invoke-RestMethod -Method Put -ContentType "application/json" -InFile {beatname_lc}.template.json -Uri http://localhost:9200/_template/{beatname_lc}-{stack-version}
|
|||
|
|
----
|