Elasticsearch hot&warm架構的簡單實現

hot&warm架構，簡單說，就是將Elasticsearch集羣部署劃分爲hot數據節點和warm數據節點，適用於time based 索引數據。其中，hot數據節點用於處理不斷有文檔新寫入的索引，warm數據節點用於處理不再有新文檔寫入，但仍有查詢需要的索引。

該架構的實現可以參考官網https://www.elastic.co/guide/en/elasticsearch/reference/7.2/shard-allocation-filtering.html 

具體實現：這裏以包含有兩個節點的集羣進行說明，node-1對應hot節點，node-2對應warm節點

1）使用node.attr標記節點

node-1的elasticsearch.yml配置爲：

cluster.name: test
node.name: node-1
path.data: /home/elk/elasticsearch-7.2.0/data
path.logs: /home/elk/elasticsearch-7.2.0/logs
http.port: 9200
network.host: 0.0.0.0
discovery.seed_hosts: ["192.168.52.127"]
cluster.initial_master_nodes: ["node-1"]
node.attr.type: hot

node-2的elasticsearch.yml配置爲：

cluster.name: test
node.name: node-2
path.data: /home/elk/elasticsearch-7.2.0/data
path.logs: /home/elk/elasticsearch-7.2.0/logs
http.port: 9200
network.host: 0.0.0.0
discovery.seed_hosts: ["192.168.52.127"]
cluster.initial_master_nodes: ["node-1"]
node.attr.type: warm

啓動node-1,node-2。在kibana中查看：GET /_cat/nodeattrs?v&h=node,attr,value

node   attr              value
node-2 ml.machine_memory 8201084928
node-2 ml.max_open_jobs  20
node-2 xpack.installed   true
node-2 type              warm
node-1 ml.machine_memory 8201076736
node-1 xpack.installed   true
node-1 ml.max_open_jobs  20
node-1 type              hot

2)創建索引到node-1節點，即hot節點

PUT logs-2020.07.05
{  
"settings": {    
"number_of_shards": 2,    
"number_of_replicas": 0,    
"index.routing.allocation.require.type": "hot"  
}
}

查看索引分佈：GET /_cat/shards/logs-2020.07.05?v&h=index,node

index           node
logs-2020.07.05 node-1
logs-2020.07.05 node-1

3）假設logs-2020.07.05不再有新數據寫入，將索引移動到node-2節點，即warm節點

PUT logs-2020.07.05/_settings
{
  "index.routing.allocation.require.type": "warm"
}

查看索引分佈：GET /_cat/shards/logs-2020.07.05?v&h=index,node

index           node
logs-2020.07.05 node-2
logs-2020.07.05 node-2