購買阿里雲ecs後所做項目五：nginx 日誌分割，格式化json日誌，日誌分析

一、日誌分割

在系統中有每天，每小時，每月，每週的定時任務文件夾，默認logrotate 是在每天執行一次，它會執行/etc/logrotate.d/下的日誌分割的配置，我們只需要創建一個nginx的配置在這裏嗎就行了

[root@iZ2ze1o ~]#ll /etc/cron
cron.d/       cron.daily/   cron.deny     cron.hourly/  cron.monthly/ crontab       cron.weekly/  
[root@iZ2ze1o ~]#cat /etc/cron.daily/logrotate 
# 以上是解釋，不需要操作，只需創建下面的文件即可
[root@iZ2ze1o ~]#vi /etc/logrotate.d/nginx 
/apps/nginx/logs/*.log {
        daily                                                                                                                
        rotate 6
        missingok
        notifempty
        create 644 nginx nginx
        postrotate
          if [ -f /apps/nginx/logs/nginx.pid ]; then
                kill -USR1 `cat /apps/nginx/logs/nginx.pid`
          fi
        endscript
}
# daily 每天
# rotate 6 保存6個文件
# missingok 忽略錯誤
# notifempty忽略空文件
# create 644 nginx nginx  創建的權限和用戶
# postrotate 分割後執行的shell
# endscript 結束shell

效果:
[root@iZ2ze1o ~]#ll /apps/nginx/logs/
total 3864
-rw-r--r-- 1 nginx nginx   12263 Jun  9 09:16 access.log
-rw-r--r-- 1 nginx nginx 2827263 Jun  8 16:07 access.log.1
-rw-r--r-- 1 root  root    13189 Jun  9 02:57 access.log-20200609
-rw-r--r-- 1 nginx nginx    6182 Jun  9 08:36 error.log
-rw-r--r-- 1 nginx nginx 1067074 Jun  8 17:25 error.log.1
-rw-r--r-- 1 nginx nginx    6738 Jun  9 02:52 error.log-20200609
-rw-r--r-- 1 root  root        4 Jun  8 14:11 nginx.pid
[root@iZ2ze1o ~]#

二、nginx 格式化json 日誌

[root@iZ2ze1o ~]#vi /apps/nginx/conf/nginx.conf
# 在http 定義日誌格式，在server 引用日誌格式
http {
    log_format access_json '{"@timestamp":"$time_iso8601",'
                           '"host":"$server_addr",'
                           '"clientip":"$remote_addr",'
                           '"size":$body_bytes_sent,'
                           '"responsetime":$request_time,'
                           '"upstreamtime":"$upstream_response_time",'
                           '"upstreamhost":"$upstream_addr",'
                           '"http_host":"$host",'
                           '"url":"$uri",'
                           '"domain":"$host",'
                           '"xff":"$http_x_forwarded_for",'
                           '"referer":"$http_referer",'
                           '"status":"$status"}';
    server {
        access_log  logs/access.log  access_json;
        }
 }
[root@iZ2ze1o ~]#nginx -s reload
[root@iZ2ze1o ~]#tail -1 /apps/nginx/logs/access.log
{"@timestamp":"2020-06-09T09:16:11+08:00","host":"xxxxx","clientip":"216.53.254.149","size":157,"responsetime":0.310,"upstreamtime":"-","upstreamhost":"-","http_host":"localhost","url":"/","domain":"localhost","xff":"-","referer":"-","status":"400"}
[root@iZ2ze1o ~]#

校驗格式是否正確

三、日誌分析

使用腳本，分析ip訪問次數，查看是否遭受某個ip 的攻擊

[root@iZ2ze1o scripts]# cat /apps/nginx/logs/access.log  | awk -F'"' '{count[$12]+=1;}END{for (i in count){print count[i]" "i;}}' | sort -t" " -k 1 -nr | head 
11 61.160.236.22
4 190.186.205.194
4 176.53.69.2
2 87.8.109.199
2 200.76.4.130
1 95.213.177.126
1 95.156.203.233
1 89.239.168.227
1 80.82.70.187
1 62.210.111.97
[root@iZ2ze1o scripts]#

因爲日誌已經分割了，下面是我以前的日誌

1. 問題一，某個ip 一直在想嘗試登陸我的WordPress

   [root@iZ2ze1o logs]#grep "//wp-login.php" access.log.1 | head
   52.187.188.65 - - [06/Jun/2020:17:15:24 +0800] "GET /wordpress//wp-login.php HTTP/1.1" 200 5251 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0 Safari/537.36"
   52.187.188.65 - - [06/Jun/2020:17:15:24 +0800] "GET /wordpress//wp-login.php HTTP/1.1" 200 5251 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0 Safari/537.36"
   52.187.188.65 - - [06/Jun/2020:17:21:21 +0800] "POST /wordpress//wp-login.php HTTP/1.1" 200 5533 "http://jiansesheying.com/wordpress//wp-login.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0 Safari/537.36"
   52.187.188.65 - - [06/Jun/2020:17:21:22 +0800] "POST /wordpress//wp-login.php HTTP/1.1" 200 5533 "http://jianseyingshi.com/wordpress//wp-login.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0 Safari/537.36"
   52.187.188.65 - - [06/Jun/2020:17:21:22 +0800] "POST /wordpress//wp-login.php HTTP/1.1" 200 5533 "http://jiansesheying.com/wordpress//wp-login.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0 Safari/537.36"
   52.187.188.65 - - [06/Jun/2020:17:21:22 +0800] "POST /wordpress//wp-login.php HTTP/1.1" 200 5533 "http://jianseyingshi.com/wordpress//wp-login.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0 Safari/537.36"
   52.187.188.65 - - [06/Jun/2020:17:21:22 +0800] "POST /wordpress//wp-login.php HTTP/1.1" 200 5533 "http://jiansesheying.com/wordpress//wp-login.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0 Safari/537.36"
   52.187.188.65 - - [06/Jun/2020:17:21:22 +0800] "POST /wordpress//wp-login.php HTTP/1.1" 200 5533 "http://jianseyingshi.com/wordpress//wp-login.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0 Safari/537.36"
   52.187.188.65 - - [06/Jun/2020:17:21:22 +0800] "POST /wordpress//wp-login.php HTTP/1.1" 200 5533 "http://jiansesheying.com/wordpress//wp-login.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0 Safari/537.36"
   52.187.188.65 - - [06/Jun/2020:17:21:23 +0800] "POST /wordpress//wp-login.php HTTP/1.1" 200 5533 "http://jianseyingshi.com/wordpress//wp-login.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0 Safari/537.36"
   [root@iZ2ze1o logs]#
   

2. 問題二：
   一些ip 使用xmlrpc.php 來查看暴力破解我的WordPress密碼

   [root@iZ2ze1o logs]#grep xmlrpc.php access.log.1 | head
   5.135.165.15 - - [23/May/2020:15:28:14 +0800] "POST /xmlrpc.php HTTP/1.1" 404 27 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"
   198.71.239.44 - - [23/May/2020:15:39:31 +0800] "POST /xmlrpc.php HTTP/1.1" 404 27 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"
   198.71.239.44 - - [23/May/2020:15:39:31 +0800] "POST /xmlrpc.php HTTP/1.1" 404 27 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"
   182.50.132.95 - - [23/May/2020:15:39:31 +0800] "POST /xmlrpc.php HTTP/1.1" 404 27 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"
   182.50.132.95 - - [23/May/2020:15:39:31 +0800] "POST /xmlrpc.php HTTP/1.1" 404 27 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"
   198.71.239.7 - - [23/May/2020:15:40:00 +0800] "POST /xmlrpc.php HTTP/1.1" 404 27 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"
   198.71.239.7 - - [23/May/2020:15:40:04 +0800] "POST /xmlrpc.php HTTP/1.1" 404 27 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"
   210.209.123.158 - - [23/May/2020:15:42:59 +0800] "POST /xmlrpc.php HTTP/1.1" 404 27 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"
   50.63.197.102 - - [23/May/2020:15:43:00 +0800] "POST /xmlrpc.php HTTP/1.1" 404 27 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"
   3.133.43.109 - - [23/May/2020:15:43:00 +0800] "POST /xmlrpc.php HTTP/1.1" 404 27 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"
   [root@iZ2ze1o logs]#
   

3. 解決辦法
   解決一：使用awk 配合iptables 來拒絕ip ，下面只是樣例，自己按需更改

   # 查看前十名ip
   cat /apps/nginx/logs/access.log  | awk -F'"' '{count[$12]+=1;}END{for (i in count){print count[i]" "i;}}' | sort -t" " -k 1 -nr | head
   
   # 查看訪問數大於100的ip
   cat /apps/nginx/logs/access.log  | awk -F'"' '{count[$12]+=1;}END{for (i in count){ if(count[i]>100){print i}}}' | sort -t" " -k 1 -nr | head
   
   #把訪問數大於100的拒絕掉
   cat /apps/nginx/logs/access.log  | awk -F'"' '{count[$12]+=1;}END{for (i in count){ if(count[i]>100){system("iptables -I INPUT -s " i " -j DROP")}}}' 
   
   # 查看上一分鐘的訪問次數
   cat /apps/nginx/logs/access.log  | awk -F'"' 'BEGIN{ now=strftime("%H:%M",systime()-60); print now;}$4 ~ now {count[$12]+=1;}END{for (i in count){print count[i]" "i;}}'
   
   # 查看上一個小時的訪問次數
   cat /apps/nginx/logs/access.log  | awk -F'"' 'BEGIN{ beg=strftime("%Y-%m-%dT%H:%M",systime()-3660); end=strftime("%Y-%m-%dT%H:%M",systime()-60); print beg; print end;}$4 > beg,$4 < end{count[$12]+=1; print $4}END{for (i in count){print count[i]" "i ;}}'   
   

   解決二：使用nginx 拒絕 xmlrpc.php的訪問

    location ~ xmlrpc.php$ {                                                                                             
               deny all;
           }