有一天突發奇想,nginx反向代理能不能記錄後端接口的處理時間呢?如果能記錄那麼就可用定位分析接口的處理異常了。
配置
如果需要nginx記錄請求時間,那麼就需要配置nginx的log_fromat
具體修改nginx.conf
配置文件
http {
include /etc/nginx/conf/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'$http_user_agent $http_x_forwarded_for $request_time $upstream_response_time $upstream_addr $upstream_status';
log_format apm '[$time_local]\tclient=$remote_addr\t'
'request="$request"\t request_length=$request_length\t'
'http_referer="$http_referer"\t'
'bytes_sent=$bytes_sent\t'
'body_bytes_sent=$body_bytes_sent\t'
'user_agent="$http_user_agent"\t'
'upstream_addr=$upstream_addr\t'
'upstream_status=$upstream_status\t'
'cookie="$http_cookie"\t'
'request_body="$request_body"\t'
'document_root="$document_root"\t'
'fastcgi_script_name="$fastcgi_script_name"\t'
'request_filename="$request_filename"\t'
'request_time=$request_time\t'
'upstream_response_time=$upstream_response_time\t'
'upstream_connect_time=$upstream_connect_time\t'
'upstream_header_time=$upstream_header_time\t';
# 爲什麼有的變量有雙引號,有的沒有呢?
# 如果變量結果是數字的,就不需要雙引號,變量是字符的,就需要雙引號
access_log /var/log/nginx/access.log main;
error_log /var/log/nginx/error.log crit;
上面配置了兩個log_format
,設置兩個log_format
的目的是爲了讓不同的後端的服務,使用不用日誌格式。
如果後端的server沒有配置,就默認使用main
標記的日誌,
access_log /var/log/nginx/access.log main; # 這裏配置的
如果需要apm
格式,只需要在access_log
配置apm即可
server {
listen 8000;
server_name www.test.com;
location / {
proxy_pass http://192.168.1.80:8080;
# 這裏設置apm即可使用apm格式的log_format
access_log /var/log/nginx/getip-access.log apm;
include /conf.d/proxy-params.conf;
}
}
結果驗證
請求 www.test.com
輸出日誌結果爲:
[11/May/2020:11:22:20 +0800]
client=192.168.1.100
request="GET / HTTP/1.1"
request_length=452
http_referer="-"
bytes_sent=170
body_bytes_sent=14
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"
upstream_addr=192.168.1.80:8080
upstream_status=200
cookie="-"
request_body="-"
document_root="/etc/nginx/html"
fastcgi_script_name="/"
request_filename="/etc/nginx/html/"
request_time=5.006
upstream_response_time=5.006
upstream_connect_time=0.000
upstream_header_time=5.006
下面爲proxy_pass http://192.168.1.80:8080;
接口處理時間,
upstream_response_time=5.006
upstream_connect_time=0.000
upstream_header_time=5.006
因爲我在後端服務器配置了sleep 5秒,因此處理時間爲5秒多,這個時間就反應了
用戶發送一次請求,到nginx代理後端服務器,再從後端服務器處理完畢,反饋給用戶的時間
記錄了一次完整的請求時間
後續可以從日誌中提取時間,分析網頁中哪個接口請求比較慢
後端服務www.test.com
的代碼
#!/bin/python
# -*- coding: UTF-8 -*-
from flask import Flask, render_template, request
import time
# Initialize the Flask application
app = Flask(__name__)
@app.route('/')
def index():
user_ip = None
if 'X-Real-Ip' in request.headers.keys():
user_ip = request.headers['X-Real-Ip']
else:
user_ip = request.remote_addr
time.sleep(5)
return user_ip
if __name__ == '__main__':
app.run(host="0.0.0.0",port=int("5000"))
後續
這個訪問日誌格式還可以優化,還可以定製不同的參數,設置爲json格式等等。
log_format main '{
"remote_addr":"$remote_addr",
"remote_user":"$remote_user",
"time_local":"$time_local",
"request":"$request",
"status":"$status",
"request_time":"$request_time",
"upstream_response_time":"$upstream_response_time",
"request_length":"$request_length",
"bytes_sent":"$bytes_sent",
"body_bytes_sent":"$body_bytes_sent",
"gzip_ratio":"$gzip_ratio",
"connection_requests":"$connection_requests",
"http_referer":"$http_referer",
"http_user_agent":"$http_user_agent",
"http_x_forwarded_for":"$http_x_forwarded_for"
}';
log_format main '{
"remote_addr":"$remote_addr",
"upstream_addr":"$upstream_addr",
"status":"$status",
"upstream_status":"$upstream_status",
"http_referer":"$http_referer",
"http_user_agent":"$http_user_agent",
"http_x_forwarded_for":"$http_x_forwarded_for"
"remote_user":"$remote_user",
"request_body":"$request_body",
"document_root":"$document_root",
"fastcgi_script_name":"$fastcgi_script_name",
"request_filename":"$request_filename",
"request":"$request",
"request_length":"$request_length",
"bytes_sent":"$bytes_sent",
"body_bytes_sent":"$body_bytes_sent",
"gzip_ratio":"$gzip_ratio",
"connection_requests":"$connection_requests",
"time_local":"$time_local",
"request_time":"$request_time",
"upstream_response_time":"$upstream_response_time",
"upstream_connect_time":"$upstream_connect_time",
"upstream_header_time":"$upstream_header_time",
}';
format變量說明
$remote_addr #發起請求的客戶端所在ip地址
$remote_user #發起請求的客戶端用戶名稱,獲取不到則顯示爲 -
$time_local #用來記錄訪問時間與時區(依賴nginx服務器本地時間),形如 20/Aug/2017:21:15:19 +0800,獲取不到則顯示爲 -
$time_iso8601 #類似$time_local,不同的是這裏採用ISO 8601標準格式
$request #記錄發起的請求,形如 POST /zentaopms/www/index.php?m=user&f=login&referer=L3plbnRhb3Btcy93d3cvaW5kZXgucGhw HTTP/1.1
$status #記錄響應狀態,比如 200
$request_time
#記錄請求處理時間(以秒爲單位,攜帶毫秒的解決方案),
#從讀取客戶端第一個字節開始算起,到發送最後一個字節給客戶端的時間間隔
#(原文:request processing time in seconds with a milliseconds resolution; time elapsed between the first bytes were read from the client and the log write after the last bytes were sent to the client)
$upstream_response_time #記錄nginx從後端服務器(upstream server)獲取響應的時間(以秒爲單位,攜帶毫秒的解決方案),多個請求的時間以逗號分隔
$request_length #記錄請求長度(包括請求行,請求頭,請求體)
$gzip_ratio #記錄nginx gzip壓縮比例,獲取不到則顯示爲 -
$bytes_sent #發送給客戶端的字節數
$body_bytes_sent #發送給客戶端的響應體字節數
$connection_requests #單個連接的併發請求數(the current number of requests made through a connection (1.1.18)
$http_referer #記錄請求引用頁面地址
$http_user_agent #記錄用戶代理信息(通常是瀏覽器信息
$http_x_forwarded_for
#當爲了承受更大的負載使用反向代理時,web服務器不能獲取真實的客戶端IP,
#$remote_addr獲取到的是反向代理服務器的ip,
#這種情況下,代理服務器通常會增加一個叫做x_forwarded_for的信息頭,
#把連接它的真實客戶端IP加到這個信息頭裏,這樣就能保證網站的web服務器能獲取到真實IP,獲取不到則顯示爲 -
$connection #連接序列號
$msec #寫入日誌的時間(以秒爲單位,攜帶毫秒的解決方案)(原文:time in seconds with a milliseconds resolution at the time of the log write)
$pipe #如果爲管道請求則顯示爲p,否則顯示爲 .
參考鏈接:
# http://nginx.org/en/docs/http/ngx_http_upstream_module.html#var_upstream_response_time
附上 nginx可用的參數大全
https://nginx.org/en/docs/http/ngx_http_core_module.html#var_status
參考鏈接:
https://www.cnblogs.com/kevingrace/p/5893499.html
https://www.cnblogs.com/shouke/p/10157556.html
http://www.ttlsa.com/linux/the-nginx-log-configuration/
https://www.cnblogs.com/taoshihan/p/11978842.html
https://blog.csdn.net/ronon77/article/details/84732095
https://www.baidu.com/s?ie=UTF-8&wd=ngx_http_log_request_speed