logstash如何從redis拉出日誌並寫入es？
input {
redis {
batch_count => 1
host => "sz-a-xxxxxredis01-redis-xen.xxxxx.com"
type => "syslog"
data_type => "list"
key => "logstash:syslog-log"
}
redis {
batch_count => 1
host => "sz-a-xxxxxredis01-redis-xen.xxxxx.com"
type => "log4j-json-web"
data_type => "list"
key => "logstash:xxxxxweb-web-log"
}
redis {
batch_count => 1
host => "sz-a-xxxxxredis01-redis-xen.xxxxx.com"
type => "log4j-json-tomcat"
data_type => "list"
key => "logstash:xxxxxweb-tomcat-log"
}
}
output{
if [type] == "syslog" {
elasticsearch { hosts => ["sz-a-xxxxxes01-es-hw.xxxxx.com"]
index => "dev-system-logs-%{+YYYY.MM.dd}"
document_type => "system-logs" }
}
if [type] == "log4j-json-web" {
elasticsearch { hosts => ["sz-a-xxxxxes01-es-hw.xxxxx.com"]
index => "dev-xxxxxweb-master-logs-%{+YYYY.MM.dd}"
document_type => "%{class}" }
}
if [type] == "log4j-json-tomcat" {
elasticsearch { hosts => ["sz-a-xxxxxes01-es-hw.xxxxx.com"]
index => "dev-xxxxxweb-master-logs-%{+YYYY.MM.dd}"
document_type => "tomcat_logs" }
}
}
以上配置中還涉及到了rsyslog日誌的傳輸，借鑑了另一位大神的配置，他的博客地址：http://dl528888.blog.51cto.com
總結
本文中最後其實少了kibana效果展示，是因爲展示圖還在完善中，在後面的記述中，會展示一些kibana呈現的效果。