ELK監控tomcat，圖形化顯示日誌統計數據

E-elasticSearch:存儲日誌數據

L-Logstash:格式化日誌文件並輸出到elasticSearch中

K-Kibana：以圖形界面的形式展示日誌信息

操作系統	Red Hat 4.8.3-9
瀏覽器	Google Chrome 79.0.3945.88
es瀏覽插件	elasticsearch-head
logstash	logstash 7.2.0
elasticSearch	elasticSearch 7.2.0
目標日誌	tomcat
java日誌格式	`log4j.appender.CONSOLE.layout.ConversionPattern=%t %d{yyyy-MM-dd HH:mm:ss} %p [%c:%L] %m%n`
INFO類型日誌	http-pre-8080-exec-6 2020-01-21 17:06:23 INFO [com.Demo.AClass.BClass.CLass:996] 開始創建演示日誌
WARN類型日誌	http-pre-8080-exec-6 2020-01-21 15:25:15 WARN [com.datastax.driver.core.Session:382] Error creating pool to /192.168.1.1:9042 com.datastax.driver.core.exceptions.ConnectionException: [/192.168.1.1:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:149) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:135) at com.google.common.util.concurrent.Futures$6.run(Futures.java:1773) at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:310) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:613) at com.google.common.util.concurrent.CollectionFuture$CollectionFutureRunningState.handleAllCompleted(CollectionFuture.java:76) at com.google.common.util.concurrent.AggregateFuture$RunningState.processCompleted(AggregateFuture.java:255) at com.google.common.util.concurrent.AggregateFuture$RunningState.decrementCountAndMaybeComplete(AggregateFuture.java:242) at com.google.common.util.concurrent.AggregateFuture$RunningState.access$300(AggregateFuture.java:91) at com.google.common.util.concurrent.AggregateFuture$RunningState$1.run(AggregateFuture.java:146) at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456)

/etc/logstash/conf.d中添加tomcat.conf日誌配置文件，內容如下

#
# usermod -a -G root logstash 授權訪問tomcat用戶的日誌
#
input{
  file{
    path => "/root/tomcat/logs/catalina.out"
    type => "tomcat-catalina"
    start_position => "beginning"
    #下面multiline的加入是爲了出現exception時統計到一行中
    codec => multiline {
          # Grok pattern names are valid! :)
          pattern => "\s*http-pre-%{DATA}"
          negate => true
          what => "previous"
    }
  }
}

filter {
  if [type] == "tomcat-catalina" {
    grok {
    patterns_dir => "/usr/local/logstash/patterns"
    match => { "message" => "%{TOMCAT_CATALINA}" }
    }
    date{
    match => ["mytimestamp", "yyyy-MM-dd HH:mm:ss"]
    target => "@timestamp"
    }
    #如果"exclude"在logmessage中，則這一整行都刪除
    if(  "exclude" in [logmessage]) {
      drop{}
    }

  mutate{
  convert => {
         "javaLineNumber" => "float"
    }
    #刪除原始的message
    remove_field => [ "message" ]
  }

  }
  
}


output{
  if [type] == "tomcat-catalina" {
      elasticsearch{
      hosts => "localhost:9200"
      user => "abc"
      password => "123"
      index => "tomcat-catalina-%{+YYYY.MM.dd}"
      template => "/usr/local/logstash/template/tomcatTemplate.json"
      template_name => "tomcat-catalina-*"
      template_overwrite => true
      manage_template => false
    }
  }
  
}

一定要記得給logslogstash用戶授權。usermod -a -G root logstash

pattern文件tomcat

JAVACLASS (?:[a-zA-Z$_][a-zA-Z$_0-9]*\.)*[a-zA-Z$_][a-zA-Z$_0-9]*:%{INT:javaLineNumber}
THREAD  ([a-zA-Z$_0-9]*\-)*([0-9])
JAVALOGMESSAGE (.*)
MYTIMESTAMP 20%{YEAR}-%{MONTHNUM}-%{MONTHDAY}\s%{HOUR}:%{MINUTE}:%{SECOND}
TOMCAT_CATALINA (%{THREAD:thread})\s(%{MYTIMESTAMP:mytimestamp})\s(%{LOGLEVEL:level})\s[[:punct:]](%{JAVACLASS:class})[[:punct:]]\s(%{JAVALOGMESSAGE:logmessage})

pattern文件的調試可以到開發工具》處進行日誌格式的匹配，

grok中已存在的正則表達式點擊 https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns

效果如下

或者登陸http://grokdebug.herokuapp.com/進行調試。

es模板文件tomcatTemplate.json

{
    "index_patterns" : [
      "tomcat-catalina-*"
    ],
    "settings" : {
      "index" : {
        "number_of_shards" : "1"
      }
    },
    "mappings" : {
      "_source" : {
        "enabled" : true
      },
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "mytimestamp" : {
          "type" : "text"
        },
        "level" : {
          "type" : "text"
        },
        "@version" : {
          "type" : "text"
        },
        "logmessage" : {
          "type" : "text"
        },
        "thread" : {
          "type" : "text"
        },
        "class" : {
          "type" : "text"
        }
      }
    }

}

執行systemctl start logstash啓動logstash，執行命令tail -f /var/log/logstash/logstash-plain.log 查看logstash是否啓動成功。

2.elasticsearch查看

填寫es的地址:端口號，用戶名/密碼