elasticsearch5.2.0和Kibana安裝詳解及問題

一、es集羣的規劃準備:
1、修改/etc/hosts文件
192.168.233.131 node1
192.168.233.132 node2
192.168.233.133 node3

臨時修改主機名
hostname node1
hostname node2
hostname node3
2、均做免密碼登陸配置
ssh-keygen -t rsa
ssh-copy-id -i /root/.ssh/id_rsa.pub node1
ssh-copy-id -i /root/.ssh/id_rsa.pub node2
ssh-copy-id -i /root/.ssh/id_rsa.pub node3
3、編寫shell

在node1機器root目錄下,產生一個nodes文件,文件內容如下,主要將各節點的主機名維護起來
node02
node03

在node1機器root目錄下,產生一個scp.sh文件,文件內容如下,輸入需要傳送的源文件路徑與目標機器的存放路徑兩個參數,路徑都是絕對路徑。

#!/bin/bash
cat /root/nodes | while read host
do
{
    scp -r $1 $host:$2
}&wait
done

4.安裝jdk1.8,將java_home加入環境變量


5.添加運行elasticsearch的用戶(es必須使用非root用戶啓動)
groupadd elsearch
useradd elsearch -g elsearch -p 123456
chown -R elsearch:elsearch /opt/elasticsearch

6.針對es做的一些系統的優化配置
swapoff -a

echo "fs.file-max = 1000000" >> /etc/sysctl.conf
echo "vm.max_map_count=655360" >> /etc/sysctl.conf
echo "vm.swappiness = 1" >> /etc/sysctl.conf
sysctl -p

sed -i 's/* soft nofile 65535/* soft nofile 655350/g' /etc/security/limits.conf
sed -i 's/* hard nofile 65535/* hard nofile 655350/g' /etc/security/limits.conf


vi /etc/security/limits.d/90-nproc.conf
* soft nproc 2048

7.es內存調整配置文件(建議配置爲物理內存的一半或者更多最好不要超過32G,超過了也可能不會增強性能):

/opt/elasticsearch/elasticsearch-5.2.0/config/jvm.options

sed -i 's/-Xms2g/-Xms32g/' /opt/elasticsearch/elasticsearch-5.2.0/config/jvm.options
sed -i 's/-Xmx2g/-Xmx32g/' /opt/elasticsearch/elasticsearch-5.2.0/config/jvm.options
echo "-Xss256k" >>/opt/elasticsearch/elasticsearch-5.2.0/config/jvm.options

sed -i 's/-XX:+UseConcMarkSweepGC/-XX:+UseG1GC/' /opt/elasticsearch/elasticsearch-5.2.0/config/jvm.options

8.集羣的主要配置文件
vim  /opt/elasticsearch/elasticsearch-5.2.0/config/elasticsearch.yml


cluster.name: es-cluster
node.name: node1
http.port: 9200
discovery.zen.ping.unicast.hosts: ["172.22.70.211","172.22.70.213","172.22.70.214"]
discovery.zen.minimum_master_nodes: 1
http.cors.enabled: true
http.cors.allow-origin: "*"
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 0.0.0.0

9、拷貝文件並啓動
sh ~/scp.sh /opt/elasticsearch/ /opt/
sh ~/scp.sh /etc/security/limits.d/90-nproc.conf /etc/security/limits.d/
sh ~/scp.sh /etc/security/limits.conf /etc/security/
sh ~/scp.sh /etc/sysctl.conf /etc/

修改各節點的/opt/elasticsearch/elasticsearch-5.2.0/config/elasticsearch.yml
node.name和端口號

# su - elsearch
cd /opt/elasticsearch/elasticsearch-5.2.0/bin/
或./elasticsearch &
8.檢查單臺服務是否正常:

$ curl http://ip:9200/
{
  "name" : "yunva_etl_es5",
  "cluster_name" : "yunva-es",
  "cluster_uuid" : "2shAg8u3SjCRNJ4mEUBzBQ",
  "version" : {
    "number" : "5.0.1",
    "build_hash" : "080bb47",
    "build_date" : "2016-11-11T22:08:49.812Z",
    "build_snapshot" : false,
    "lucene_version" : "6.2.1"
  },
  "tagline" : "You Know, for Search"
}


# 查看集羣狀態
$ curl http://ip:9200/_cluster/health/?pretty
{
  "cluster_name" : "yunva-es",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 5,
  "number_of_data_nodes" : 4,
  "active_primary_shards" : 66,
  "active_shards" : 132,
  "relocating_shards" : 2,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}


然後將配置好的es程序拷貝到其他服務器中,注意修改以下內容(network.host爲內網地址,速度更快,節省互相複製、分片的時候處理帶寬):
1.elasticsearch.yml文件的配置修改
node.name: 節點名稱
network.host: es節點的內網IP地址
2./etc/hosts文件中內網ip和node.name的對應關係

    
三、安裝Kibana5.2.0
1、vim kibana/config/kibana.yml
server.port: 5601
server.host: "本機IP"
elasticsearch.url: "http://本機IP:9200"
2、elasticsearch/bin/elasticsearch  //啓動ES
kibana/bin/kibana                      //啓動kibana
curl -XGET http://192.168.230.150:9200/ 結果如下則成功。

    {
      "name" : "node-1",
      "cluster_name" : "es",
      "cluster_uuid" : "bbCPwel7Tn-1cip2rsFWRQ",
      "version" : {
        "number" : "5.3.1",
        "build_hash" : "5f9cf58",
        "build_date" : "2017-04-17T15:52:53.846Z",
        "build_snapshot" : false,
        "lucene_version" : "6.4.2"
      },
      "tagline" : "You Know, for Search"
    }

瀏覽器訪問9200:http://192.168.230.150:9200可得到上一步同樣結果。

瀏覽器訪問5601:http://192.168.230.150:5601.第一次Kibana會進入創建索引的界面,這裏創建名爲*的索引匹配數據(由於開始沒有數據)。(取消掉所有打鉤的地方)點擊Create成功會顯示在左側。可以看到所有的功能組件。


三、Elasticsearch,Kibana安裝中的異常解決:

1、[ WARN]:seccomp unavailable:requires kernel 3.5+ with CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER compiled in。

    警告內核版本太低!,忽略警告。

2、[ERROR]:max number of threads [1024] for user [rzxes] is too low, increase to at least [2048]。

    sudo vim /etc/security/limits.d/90-nproc.conf 把1024改成2048。

3、[ERROR]:max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]。

    Sudo vim /etc/security/limits.conf ,末尾添加如下兩行:
    * hard nofile 65536
    * soft nofile 65536
    查看: ulimit -Hn 結果是65536則修改成功。

4、[ERROR]:system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk。

    vim config/elasticsearch.yml 添加如下:
    bootstrap.system_call_filter: false

5、[ERROR]:max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]。

    修改/etc/sysctl.conf sudo vim /etc/sysctl.conf ,最後添加: vm.max_map_count=262144 。
    使用 sysctl -p 查看修改後的結果。
6、[2017-09-15T14:03:19,153][INFO ][o.e.d.z.ZenDiscovery     ] [node3] failed to send join request to master [{node1}{Zo4L-JnrTFq50lsHaH-wsQ}{b4S6zrexQSenOM1bqJCBfg}{172.22.70.211}{172.22.70.211:9300}], reason [RemoteTransportException[[node1][172.22.70.211:9300][internal:discovery/zen/join]]; nested: NotMasterException[Node [{node1}{Zo4L-JnrTFq50lsHaH-wsQ}{b4S6zrexQSenOM1bqJCBfg}{172.22.70.211}{172.22.70.211:9300}] not master for join request]; ], tried [3] times

刪除data/ 和 logs/下的所有文件 後重新啓動即可,產生如下結果爲正確
[2017-09-15T15:35:49,375][INFO ][o.e.c.s.ClusterService   ] [node1] new_master {node1}{VfQ4RZ_eQX-32cTNDCJD1w}{AjZOBuEcQlmy_6yDXorBaQ}{172.22.70.211}{172.22.70.211:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-09-15T15:35:49,436][INFO ][o.e.h.HttpServer         ] [node1] publish_address {172.22.70.211:9200}, bound_addresses {[::]:9200}
[2017-09-15T15:35:49,437][INFO ][o.e.n.Node               ] [node1] started
[2017-09-15T15:35:49,467][INFO ][o.e.g.GatewayService     ] [node1] recovered [0] indices into cluster_state
[2017-09-15T15:38:20,307][INFO ][o.e.c.s.ClusterService   ] [node1] added {{node2}{zs1yhSwZRyqpawOmOI5Wag}{XyOQ-G4ESEizWkh3kGsRPw}{172.22.70.213}{172.22.70.213:9300},}, reason: zen-disco-node-join[{node2}{zs1yhSwZRyqpawOmOI5Wag}{XyOQ-G4ESEizWkh3kGsRPw}{172.22.70.213}{172.22.70.213:9300}]
[2017-09-15T15:38:20,451][WARN ][o.e.d.z.ElectMasterService] [node1] value for setting "discovery.zen.minimum_master_nodes" is too low. This can result in data loss! Please set it to at least a quorum of master-eligible nodes (current value: [1], total number of master-eligible nodes used for publishing in this round: [2])
[2017-09-15T15:39:06,512][INFO ][o.e.c.s.ClusterService   ] [node1] added {{node3}{BvdBNID7QC2nUc3JoWLxSQ}{JL3M1zJyTne_gkkPhNM2Tg}{172.22.70.214}{172.22.70.214:9300},}, reason: zen-disco-node-join[{node3}{BvdBNID7QC2nUc3JoWLxSQ}{JL3M1zJyTne_gkkPhNM2Tg}{172.22.70.214}{172.22.70.214:9300}]




發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章