實驗環境爲2臺測試機分別爲Server201(192.168.10.201)和Server202(192.168.19.202),分別安裝好fluentd,具體安裝步驟不再累述。
重點描述2臺機器上的配置,目的是將Server202上的日誌,通過fluentd橋接的方式,放置到Server201下的某個目錄。
1、Server202上fluentd.conf:
<source>
type tail
#需要轉儲的日誌1
path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001.log
format /^*(?<message>.*)$/
tag mongo.apache
</source>
<source>
type tail
#需要轉儲的日誌2
path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1002.log
format /^*(?<message>.*)$/
tag mongo.apache
</source>
源日誌爲2個日誌文件。分別爲path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001.log和path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1002.log
標籤爲tag mongo.apache,方便match段匹配。
<match mongo.**>
type forward
#轉儲服務器
host 192.168.19.201
<secondary>
host 192.168.19.201
</secondary>
</match>
分別表示了2臺機器,可以添加備用機,這裏的主、備機IP均相同,生產環境可以添加不同的IP。
2、Server201上fluent.conf配置
爲了方便測試,我們直接修改,將註釋直接打開,其含義爲匹配沒有match的日誌,寫入到文件。
<match **>
type file
#存儲路徑
path /var/log/fluent/else
compress gz
</match>
啓動fluentd
201上查看配置的路徑
[root@Server201 fluentd-0.10.6]# ll /var/log/fluent/
total 68
-rw-r--r-- 1 root root 61675 Sep 28 10:20 else.20120928.b4cab973e8d931ee9
產生了文件else.20120928.b4cab973e8d931ee9
3、啓動Server202下的fluentd:fluentd -c fluent.conf -deamon -o log/log.log
tail 輸入日誌,得到後幾行輸出
2012-09-28 09:58:45 +0800: adding match pattern="mongo.**" type="forward"
2012-09-28 09:58:45 +0800: 'host' option in forward output is obsoleted. Use '<server> host xxx </server>' instead.
2012-09-28 09:58:45 +0800: adding forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224 weight=60
2012-09-28 09:58:45 +0800: 'host' option in forward output is obsoleted. Use '<server> host xxx </server>' instead.
2012-09-28 09:58:45 +0800: adding forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224 weight=60
2012-09-28 09:58:45 +0800: listening fluent socket on 0.0.0.0:24224
2012-09-28 09:58:45 +0800: following tail of /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001.log
2012-09-28 09:58:45 +0800: following tail of /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1002.log
標記顏色的2行表示2臺接受數據的機子,端口爲24224,
開始發送數據的時候這個日誌還會得到如下輸出
2012-09-28 10:02:07 +0800: detached forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224 phi=8.280932634573869
2012-09-28 10:02:07 +0800: detached forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224 phi=8.280785193737733
2012-09-28 10:02:42 +0800: recovered forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224
2012-09-28 10:02:42 +0800: recovered forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224
tail -f /var/log/fluent/else.20120928.b4cab973e8d931ee9
大概顯示如下輸出,日誌刷新頻次爲1分鐘
2012-09-28 10:23:09 +0800: plugin/in_forward.rb:140:initialize: accepted fluent socket object_id=242397940
2012-09-28 10:23:09 +0800: plugin/in_forward.rb:181:on_close: closed fluent socket object_id=242397940
2012-09-28T10:23:09+08:00 fluent.trace {"message":"accepted fluent socket object_id=242397940"}
2012-09-28T10:22:23+08:00 mongo.apache {"message":"Fri Sep 28 10:22:23 [conn369] end connection 192.168.19.201:51962"}
2012-09-28T10:22:23+08:00 mongo.apache {"message":"Fri Sep 28 10:22:23 [initandlisten] connection accepted from 192.168.19.201:51968 #372"}
2012-09-28T10:22:27+08:00 mongo.apache {"message":"Fri Sep 28 10:22:27 [conn370] end connection 192.168.19.201:51963"}
2012-09-28T10:22:27+08:00 mongo.apache {"message":"Fri Sep 28 10:22:27 [initandlisten] connection accepted from 192.168.19.201:51969 #373"}
2012-09-28T10:22:33+08:00 mongo.apache {"message":"Fri Sep 28 10:22:33 [conn371] end connection 192.168.19.202:39038"}
2012-09-28T10:22:33+08:00 mongo.apache {"message":"Fri Sep 28 10:22:33 [initandlisten] connection accepted from 192.168.19.202:39045 #374"}
2012-09-28T10:22:34+08:00 mongo.apache {"message":"Fri Sep 28 10:22:34 [conn378] end connection 192.168.19.202:55169"}
2012-09-28T10:22:34+08:00 mongo.apache {"message":"Fri Sep 28 10:22:34 [initandlisten] connection accepted from 192.168.19.202:55176 #381"}
2012-09-28T10:22:35+08:00 mongo.apache {"message":"Fri Sep 28 10:22:35 [conn379] end connection 192.168.19.201:41497"}
2012-09-28T10:22:35+08:00 mongo.apache {"message":"Fri Sep 28 10:22:35 [initandlisten] connection accepted from 192.168.19.201:41503 #382"}
2012-09-28T10:22:37+08:00 mongo.apache {"message":"Fri Sep 28 10:22:37 [conn380] end connection 192.168.19.201:41498"}
2012-09-28T10:22:37+08:00 mongo.apache {"message":"Fri Sep 28 10:22:37 [initandlisten] connection accepted from 192.168.19.201:41504 #383"}
2012-09-28T10:22:53+08:00 mongo.apache {"message":"Fri Sep 28 10:22:53 [conn372] end connection 192.168.19.201:51968"}
2012-09-28T10:22:53+08:00 mongo.apache {"message":"Fri Sep 28 10:22:53 [initandlisten] connection accepted from 192.168.19.201:51974 #375"}
2012-09-28T10:22:56+08:00 mongo.apache {"message":"Fri Sep 28 10:22:56 [clientcursormon] mem (MB) res:76 virt:419 mapped:224"}
2012-09-28T10:22:57+08:00 mongo.apache {"message":"Fri Sep 28 10:22:57 [conn373] end connection 192.168.19.201:51969"}
2012-09-28T10:22:57+08:00 mongo.apache {"message":"Fri Sep 28 10:22:57 [initandlisten] connection accepted from 192.168.19.201:51975 #376"}
2012-09-28T10:23:01+08:00 mongo.apache {"message":"Fri Sep 28 10:23:01 [clientcursormon] mem (MB) res:44 virt:329 mapped:160"}
2012-09-28T10:23:03+08:00 mongo.apache {"message":"Fri Sep 28 10:23:03 [conn374] end connection 192.168.19.202:39045"}
2012-09-28T10:23:03+08:00 mongo.apache {"message":"Fri Sep 28 10:23:03 [initandlisten] connection accepted from 192.168.19.202:39051 #377"}
2012-09-28T10:23:04+08:00 mongo.apache {"message":"Fri Sep 28 10:23:04 [conn381] end connection 192.168.19.202:55176"}
2012-09-28T10:23:04+08:00 mongo.apache {"message":"Fri Sep 28 10:23:04 [initandlisten] connection accepted from 192.168.19.202:55182 #384"}
2012-09-28T10:23:05+08:00 mongo.apache {"message":"Fri Sep 28 10:23:05 [conn382] end connection 192.168.19.201:41503"}
2012-09-28T10:23:05+08:00 mongo.apache {"message":"Fri Sep 28 10:23:05 [initandlisten] connection accepted from 192.168.19.201:41509 #385"}
2012-09-28T10:23:07+08:00 mongo.apache {"message":"Fri Sep 28 10:23:07 [conn383] end connection 192.168.19.201:41504"}
2012-09-28T10:23:07+08:00 mongo.apache {"message":"Fri Sep 28 10:23:07 [initandlisten] connection accepted from 192.168.19.201:41510 #386"}
2012-09-28T10:23:09+08:00 fluent.trace {"message":"closed fluent socket object_id=242397940"}
至此已經完全實現2臺服務器上2個fluentd的橋接。