Elastic Search

Elastic Search maintenance: Proper restart of ES nodes: https://www.elastic.co/guide/en/elasticsearch/guide/current/_rolling_restarts.html

Step 1: Disable shard allocation:

When you shut down a node, the allocation process will immediately try to replicate the shards that were on that node to other nodes in the cluster, causing a lot of wasted I/O. This can be avoided by disabling allocation before shutting down a node:

host=localhost:9200
curl -XPUT -H'Content-Type: application/json' http://$host/_cluster/settings?pretty=true -d '{"persistent": {"cluster.routing.allocation.enable": "none"}}'

Step 2: Perform a synced flush Shard recovery will be much faster if you stop indexing and issue a synced-flush request:

curl -XPOST -H'Content-Type: application/json' http://$host/_flush/synced?pretty=true

Step 3: after the restart is done, enable shard allocation again:

curl -XPUT -H'Content-Type: application/json' http://$host/_cluster/settings?pretty=true -d '{"persistent": {"cluster.routing.allocation.enable": "all"}}'

[osvadkovsky@elk-search-0901 ~]$ curl -s "localhost:9200/_all?filter_path=*.settings.index.number_of_*&pretty=true" | head -22
{
  ".kibana_1" : {
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "number_of_replicas" : "1"
      }
    }
  },
  ".monitoring-es-6-2019.02.16" : {
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "number_of_replicas" : "1"
      }
    }
  },
  ".monitoring-es-6-2019.02.17" : {
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "number_of_replicas" : "1"