filebeat logstash配置模板說明
爲了讓不同類型的日誌記錄到不同index，實現日誌分類，需要更改默認的配置文件，ELK更新迭代速度很快，網上以前的文檔適用於之前的版本
filebeat的docment_type配置項已經在6版本中棄用，請使用本文配置
filebeat配置模板
deploy@Product1:~$ cat /etc/filebeat/filebeat.yml filebeat.prospectors: - type: log enabled: true paths: - /var/www/bigbear_server/shared/log/ms.log fields: log_topics: server - type: log enabled: true paths: - /var/www/bigbear_websocket/shared/log/ms.log fields: log_topics: socket - type: log enabled: true paths: - /var/www/bigbear_admin/shared/log/ms.log fields: log_topics: admin output.logstash: hosts: ["74.xxx.xx.xx:5044"] deploy@Product3:~$ cat /etc/filebeat/filebeat.yml filebeat.prospectors: - type: log enabled: true paths: - /var/www/bigbear_server/shared/log/ms.log fields: log_topics: server - type: log enabled: true paths: - /var/www/bigbear_sidekiq/shared/log/ms.log fields: log_topics: sidekiq - type: log enables: true paths: - /application/nginx/logs/access810*.log - /application/nginx/logs/access.log 日誌的位置 fields: log_topics: nginx 日誌註明類型(logstash爲不同類型創建不同索引) output.logstash: hosts: ["74.xx.xx.xx:5044"]
logstash配置模板
deploy@Product4:~$ cat /application/logstash-6.2.4/config/02-beats-input.conf
input {
beats {
# host => "74.207.240.124"
codec => plain{ charset => "UTF-8" }
port => 5044
# ssl => true
# ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
# ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}
output {
elasticsearch {
codec => plain{ charset => "UTF-8" }
hosts => "http://localhost:9200"
user => "elastic"
password => "OFBqKJ8XjSWlX8AWL0xs"
manage_template => false
index => "%{[fields][log_topics]}--%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}重新啓動 kibana和 filebeat
可以看到 每天都會生成一個索引按照定義的fields
