1 .概述
ELK套件(ELK stack)是指ElasticSearch、Logstash和Kibana三件套。這三個軟件可以組成一套日誌分析和監控工具。
由於三個軟件各自的版本號太多,建議採用ElasticSearch官網推薦的搭配組合:http://www.elasticsearch.org/overview/elkdownloads/
2 .環境準備
2.1 軟件要求
本文把ELK套件部署在一臺CentOS單機上。
具體的版本要求如下:
操作系統版本:CentOS 6.4;
JDK版本:1.7.0;
Logstash版本:1.4.2;
ElasticSearch版本:1.4.2;
Kibana版本:3.1.2;
2.2 防火牆配置
爲了正常使用HTTP服務等,需要關閉防火牆:
# service iptables stop
# service iptables stop
或者可以不關閉防火牆,但是要在iptables中打開相關的端口:
# vim /etc/sysconfig/iptables
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 9200 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 9292 -j ACCEPT
# service iptables restart
# vim /etc/sysconfig/iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 9200 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 9292 -j ACCEPT # service iptables restart
3.安裝JDK
ElasticSearch和Logstash依賴於JDK,所以需要安裝JDK:
# yum -y install java-1.7.0-openjdk*
# java -version
# yum -y install java-1.7.0-openjdk* # java -version
4. 安裝ElasticSearch
ElasticSearch默認的對外服務的HTTP端口是9200,節點間交互的TCP端口是9300。
下載ElasticSearch:
# mkdir -p /opt/software && cd /opt/software
# sudo wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.4.2.tar.gz
# sudo tar -zxvf elasticsearch-1.4.2.tar.gz -C /usr/local/
# ln -s /usr/local/elasticsearch-1.4.2 /usr/local/elasticsearch
# mkdir -p /opt/software && cd /opt/software # sudo wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.4.2.tar.gz # sudo tar -zxvf elasticsearch-1.4.2.tar.gz -C /usr/local/ # ln -s /usr/local/elasticsearch-1.4.2 /usr/local/elasticsearch
安裝elasticsearch-servicewrapper,並啓動ElasticSearch服務:
# sudo wget https://github.com/elasticsearch/elasticsearch-servicewrapper/archive/master.tar.gz
# sudo tar -zxvf master
# mv /opt/software/elasticsearch-servicewrapper-master/service /usr/local/elasticsearch/bin/
# /usr/local/elasticsearch/bin/service/elasticsearch start
# sudo wget https://github.com/elasticsearch/elasticsearch-servicewrapper/archive/master.tar.gz # sudo tar -zxvf master # mv /opt/software/elasticsearch-servicewrapper-master/service /usr/local/elasticsearch/bin/ # /usr/local/elasticsearch/bin/service/elasticsearch start
測試ElasticSearch服務是否正常,預期返回200的狀態碼:
# curl -X GET http://localhost:9200
# curl -X GET http://localhost:9200
5.安裝Logstash
Logstash默認的對外服務的端口是9292
下載Logstash:
# sudo wget https://download.elasticsearch.org/logstash/logstash/logstash-1.4.2.tar.gz
# sudo tar -zxvf logstash-1.4.2.tar.gz -C /usr/local/
# ln -s /usr/local/logstash-1.4.2 /usr/local/logstash
# sudo wget https://download.elasticsearch.org/logstash/logstash/logstash-1.4.2.tar.gz # sudo tar -zxvf logstash-1.4.2.tar.gz -C /usr/local/ # ln -s /usr/local/logstash-1.4.2 /usr/local/logstash
簡單測試Logstash服務是否正常,預期可以將輸入內容以簡單的日誌形式打印在界面上:
# /usr/local/logstash/bin/logstash -e 'input { stdin { } } output { stdout {} }'
# /usr/local/logstash/bin/logstash -e 'input { stdin { } } output { stdout {} }'
創建Logstash配置文件,並再次測試Logstash服務是否正常,預期可以將輸入內容以結構化的日誌形式打印在界面上:
# mkdir -p /usr/local/logstash/etc
# vim /usr/local/logstash/etc/hello_search.conf
input {
stdin {
type => "human"
}
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
host => "10.111.121.22"
port => 9300
}
}
# /usr/local/logstash/bin/logstash -f /usr/local/logstash/etc/hello_search.conf
# mkdir -p /usr/local/logstash/etc # vim /usr/local/logstash/etc/hello_search.conf input { stdin { type => "human" } } output { stdout { codec => rubydebug } elasticsearch { host => "10.111.121.22" port => 9300 } } # /usr/local/logstash/bin/logstash -f /usr/local/logstash/etc/hello_search.conf
6.安裝Kibana
CentOS默認預裝了Apache,所以將Kibana的代碼直接拷貝到Apache可以訪問的目錄下即可
# sudo wget https://download.elasticsearch.org/kibana/kibana/kibana-3.1.2.tar.gz
# sudo tar -zxvf kibana-3.1.2.tar.gz
# mv kibana-3.1.2 /var/www/html/kibana
# sudo wget https://download.elasticsearch.org/kibana/kibana/kibana-3.1.2.tar.gz # sudo tar -zxvf kibana-3.1.2.tar.gz # mv kibana-3.1.2 /var/www/html/kibana
修改Kibana的配置文件,把elasticsearch所在行的內容替換成如下:
# vim /var/www/html/kibana/config.js
elasticsearch: "http://10.111.121.22:9200",
# vim /var/www/html/kibana/config.js elasticsearch: "http://10.111.121.22:9200",
啓動一下HTTP服務:
# service httpd start
# service httpd start
修改ElasticSearch的配置文件,追加一行內容,並重啓ElasticSearch服務:
# vim /usr/local/elasticsearch/config/elasticsearch.yml
http.cors.enabled: true
# /usr/local/elasticsearch/bin/service/elasticsearch restart
# vim /usr/local/elasticsearch/config/elasticsearch.yml http.cors.enabled: true # /usr/local/elasticsearch/bin/service/elasticsearch restart
然後就可以通過瀏覽器訪問Kibana了:
http://10.111.121.22/kibana
http://10.111.121.22/kibana
現在,在之前的Logstash會話中輸入任意字符,就可以在Kibana中查看到日誌情況。
7. 配置Logstash
再次創建Logstash配置文件,這裏將HTTP日誌和文件系統日誌作爲輸入,輸出直接傳給ElasticSearch,不再打印在界面上:
# vim /usr/local/logstash/etc/logstash_agent.conf
input {
file {
type => "http.access"
path => ["/var/log/httpd/access_log"]
}
file {
type => "http.error"
path => ["/var/log/httpd/error_log"]
}
file {
type => "messages"
path => ["/var/log/messages"]
}
}
output {
elasticsearch {
host => "10.111.121.22"
port => 9300
}
}
# nohup /usr/local/logstash/bin/logstash -f /usr/local/logstash/etc/logstash_agent.conf &
# vim /usr/local/logstash/etc/logstash_agent.conf input { file { type => "http.access" path => ["/var/log/httpd/access_log"] } file { type => "http.error" path => ["/var/log/httpd/error_log"] } file { type => "messages" path => ["/var/log/messages"] } } output { elasticsearch { host => "10.111.121.22" port => 9300 } } # nohup /usr/local/logstash/bin/logstash -f /usr/local/logstash/etc/logstash_agent.conf &