[[upgrading]]
== Upgrading

This section gives general recommendations for upgrading the Beats:

* <<upgrading-minor-versions>>
* <<upgrading-5-to-6>>
* <<upgrade-mapping-template>>

If you're upgrading other products in the stack, also read the
{stack-ref}/index.html[Elastic Stack Installation and Upgrade Guide]. Want an
upgrade list that's tailored to your stack? Try out our
{upgrade_guide}[Interactive Upgrade Guide].

[[upgrading-minor-versions]]
=== Upgrade between minor versions

As a general rule, upgrading between minor versions (e.g. 6.x to 6.y, where x <
y) can be done by simply installing the new release and restarting the Beat
process, because the Beats keep backwards compatibility when it comes to
configuration and exported fields. Please review the <<release-notes,release
notes>> for potential exceptions.

Upgrading between non-consecutive major versions (e.g. 1.x to 6.x) is not
supported. If you are on version 1.x of the Beats, we recommend first upgrading
to 5.6.

[[upgrading-5-to-6]]
=== Upgrade from 5.x to 6.x

Before upgrading your Beats, please review the <<breaking-changes, breaking
changes>> doc.

// TODO: link to the stack upgrade docs
//
// If you are planning an upgrade of the full stack (Elasticsearch, Kibana, or
// Logstash, in addition to Beats), please read the stack upgrade guide.

We recommend that you fully upgrade Elasticsearch and Kibana to version 6.0
before upgrading Beats. If you are on a Beats 5.x version lower than 5.6, please
follow the <<upgrading-to-5.6>> section *before* doing the Elasticsearch
upgrade.

If you're not ready to upgrade Elasticsearch and Kibana to 6.0, that's alright.
Beats version 6.0 works with Elasticsearch and Kibana version 5.6, so you can
upgrade Beats now and the rest of the stack later.

[[upgrading-to-5.6]]
==== Upgrade to 5.6

The upgrade procedure assumes that you have Beats version 5.6 installed. If you
are on a previous 5.x version of Beats, please upgrade to version 5.6 first.
The reason is that the Elasticsearch mapping template in 5.6 was modified to be
compatible with Elasticsearch 6.0 (by removing the `_all` settings).

For this reason, after upgrading to 5.6, you need to make sure that the 5.6
template is loaded. You can do this by temporarily enabling the
`output.elasticsearch.template.overwrite` setting, for example with Metricbeat:

[source,shell]
------------------------------------------------------------------------------
metricbeat -e -E output.elasticsearch.template.overwrite=true
------------------------------------------------------------------------------

Alternatively, you can manually force loading the template:

[source,shell]
------------------------------------------------------------------------------
curl -XPUT -H'Content-Type: application/json'  http://localhost:9200/_template/metricbeat -d @metricbeat.template.json
------------------------------------------------------------------------------

To check which version of the template is loaded, open Kibana Console, call `GET
/_template/metricbeat`, and look for the version string. Note that you need to
do this for each Beat type that you are running (e.g. Filebeat, Metricbeat,
Packetbeat).

==== Migrate configuration files

Beats 6.0 comes with several backwards incompatible configuration changes.
Please review the <<breaking-changes-6.0>> document. Where possible, we kept the
old configuration options working, but deprecated them.  However, deprecation
was not always possible, so if you use any of the settings mentioned in the
Breaking Changes section of the release notes, make sure you understand the
alternatives that we provide.

===== modules.d configuration layout

Starting with the 6.0 version, Filebeat and Metricbeat are moving to a directory
layout for configuration, where each module is configured in its own
configuration file.

While the all-in-one configuration is still fully supported, we recommend moving
to the new layout at upgrade time. This typically means starting off with the new
default configuration and modifying it with the custom settings that you had in
your old files.

// TODO: link to the docs on modules.d

===== New command `test config` command

Beats 6.0 introduces a new test command for testing the configuration file. For
example:

[source,shell]
------------------------------------------------------------------------------
metricbeat test config
------------------------------------------------------------------------------

The old `-configtest` flag is still available, but deprecated.

===== Reference configuration files

The `<beatname>.full.yml` file, which contains all the non-deprecated
configuration options is renamed to `<beatname>.reference.yml` starting with
Beats 6.0. We recommend using this file as a reference only. It's not intended
to be used in production.

==== Dashboard upgrades

We recommend re-importing the Kibana dashboards after the Beats and Kibana
upgrades are complete. This way, you will take advantage of the new Beats
dashboards created for the 6.0 release.

See <<breaking-changes-import-dashboards>> in the Breaking Changes
documentation.

NOTE: If you've modified the Beats dashboards manually, reimporting them will
overwrite your changes. Consider saving your dashboards under new IDs, so
that the reimport doesn't overwrite them.

==== Heartbeat package names

The DEB and RPM packages for Heartbeat are now named `heartbeat-elastic` to
avoid conflicts with a different `heartbeat` project.  The `.deb` and `.rpm`
file names haven't changed, just the package name in the repositories.

[[upgrade-mapping-template]]
=== Upgrade the Elasticsearch mapping template

Starting with Beats 6.0, the mapping templates and the default index names are
versioned. For example, Metricbeat 6.0.0 creates indices like this:

[source,shell]
------------------------------------------------------------------------------
metricbeat-6.0.0-2017-08-31
------------------------------------------------------------------------------

And the corresponding Elasticsearch template is named `metricbeat-6.0.0`.

This means that each version of the Beat creates a new index, and it is
guaranteed that the correct template for that version is applied.  With these
changes in place, you generally don't have to do anything to upgrade the mapping
template when you move to a new version.

One exception is when upgrading from 5.x, in which case you should make sure to
read the <<upgrading-to-5.6>> section.

If downtime is not acceptable, another possible approach is to configure a
different index pattern in the new Beat version, but this will likely require
adjustments to your Kibana dashboards.
