Redhat linux 下 fluentd+fluentd 實現橋接

實驗環境爲2臺測試機分別爲Server201(192.168.10.201)和Server202(192.168.19.202),分別安裝好fluentd,具體安裝步驟不再累述。

重點描述2臺機器上的配置,目的是將Server202上的日誌,通過fluentd橋接的方式,放置到Server201下的某個目錄。

1、Server202上fluentd.conf:

<source>
 type tail 

#需要轉儲的日誌1
 path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001.log
format /^*(?<message>.*)$/
 tag mongo.apache
</source>
<source>
 type tail

#需要轉儲的日誌2
 path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1002.log
format /^*(?<message>.*)$/
 tag mongo.apache
</source>

源日誌爲2個日誌文件。分別爲path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001.log和path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1002.log

標籤爲tag mongo.apache,方便match段匹配。

<match mongo.**>
  type forward

#轉儲服務器
  host 192.168.19.201
  <secondary>
    host 192.168.19.201
  </secondary>
</match>

分別表示了2臺機器,可以添加備用機,這裏的主、備機IP均相同,生產環境可以添加不同的IP。


2、Server201上fluent.conf配置

爲了方便測試,我們直接修改,將註釋直接打開,其含義爲匹配沒有match的日誌,寫入到文件。

<match **>
  type file

#存儲路徑
  path /var/log/fluent/else
  compress gz
</match>

啓動fluentd

201上查看配置的路徑

[root@Server201 fluentd-0.10.6]# ll  /var/log/fluent/
total 68
-rw-r--r-- 1 root root 61675 Sep 28 10:20 else.20120928.b4cab973e8d931ee9

產生了文件else.20120928.b4cab973e8d931ee9

3、啓動Server202下的fluentd:fluentd -c fluent.conf -deamon -o log/log.log

tail 輸入日誌,得到後幾行輸出

2012-09-28 09:58:45 +0800: adding match pattern="mongo.**" type="forward"
2012-09-28 09:58:45 +0800: 'host' option in forward output is obsoleted. Use '<server> host xxx </server>' instead.
2012-09-28 09:58:45 +0800: adding forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224 weight=60
2012-09-28 09:58:45 +0800: 'host' option in forward output is obsoleted. Use '<server> host xxx </server>' instead.
2012-09-28 09:58:45 +0800: adding forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224 weight=60
2012-09-28 09:58:45 +0800: listening fluent socket on 0.0.0.0:24224
2012-09-28 09:58:45 +0800: following tail of /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001.log
2012-09-28 09:58:45 +0800: following tail of /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1002.log

標記顏色的2行表示2臺接受數據的機子,端口爲24224,

開始發送數據的時候這個日誌還會得到如下輸出

2012-09-28 10:02:07 +0800: detached forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224 phi=8.280932634573869
2012-09-28 10:02:07 +0800: detached forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224 phi=8.280785193737733
2012-09-28 10:02:42 +0800: recovered forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224
2012-09-28 10:02:42 +0800: recovered forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224

4、查看Server201上橋接轉儲後的日誌

tail -f /var/log/fluent/else.20120928.b4cab973e8d931ee9

大概顯示如下輸出,日誌刷新頻次爲1分鐘

2012-09-28 10:23:09 +0800: plugin/in_forward.rb:140:initialize: accepted fluent socket object_id=242397940
2012-09-28 10:23:09 +0800: plugin/in_forward.rb:181:on_close: closed fluent socket object_id=242397940
2012-09-28T10:23:09+08:00       fluent.trace    {"message":"accepted fluent socket object_id=242397940"}
2012-09-28T10:22:23+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:23 [conn369] end connection 192.168.19.201:51962"}
2012-09-28T10:22:23+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:23 [initandlisten] connection accepted from 192.168.19.201:51968 #372"}
2012-09-28T10:22:27+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:27 [conn370] end connection 192.168.19.201:51963"}
2012-09-28T10:22:27+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:27 [initandlisten] connection accepted from 192.168.19.201:51969 #373"}
2012-09-28T10:22:33+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:33 [conn371] end connection 192.168.19.202:39038"}
2012-09-28T10:22:33+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:33 [initandlisten] connection accepted from 192.168.19.202:39045 #374"}
2012-09-28T10:22:34+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:34 [conn378] end connection 192.168.19.202:55169"}
2012-09-28T10:22:34+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:34 [initandlisten] connection accepted from 192.168.19.202:55176 #381"}
2012-09-28T10:22:35+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:35 [conn379] end connection 192.168.19.201:41497"}
2012-09-28T10:22:35+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:35 [initandlisten] connection accepted from 192.168.19.201:41503 #382"}
2012-09-28T10:22:37+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:37 [conn380] end connection 192.168.19.201:41498"}
2012-09-28T10:22:37+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:37 [initandlisten] connection accepted from 192.168.19.201:41504 #383"}
2012-09-28T10:22:53+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:53 [conn372] end connection 192.168.19.201:51968"}
2012-09-28T10:22:53+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:53 [initandlisten] connection accepted from 192.168.19.201:51974 #375"}
2012-09-28T10:22:56+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:56 [clientcursormon] mem (MB) res:76 virt:419 mapped:224"}
2012-09-28T10:22:57+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:57 [conn373] end connection 192.168.19.201:51969"}
2012-09-28T10:22:57+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:57 [initandlisten] connection accepted from 192.168.19.201:51975 #376"}
2012-09-28T10:23:01+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:01 [clientcursormon] mem (MB) res:44 virt:329 mapped:160"}
2012-09-28T10:23:03+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:03 [conn374] end connection 192.168.19.202:39045"}
2012-09-28T10:23:03+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:03 [initandlisten] connection accepted from 192.168.19.202:39051 #377"}
2012-09-28T10:23:04+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:04 [conn381] end connection 192.168.19.202:55176"}
2012-09-28T10:23:04+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:04 [initandlisten] connection accepted from 192.168.19.202:55182 #384"}
2012-09-28T10:23:05+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:05 [conn382] end connection 192.168.19.201:41503"}
2012-09-28T10:23:05+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:05 [initandlisten] connection accepted from 192.168.19.201:41509 #385"}
2012-09-28T10:23:07+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:07 [conn383] end connection 192.168.19.201:41504"}
2012-09-28T10:23:07+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:07 [initandlisten] connection accepted from 192.168.19.201:41510 #386"}
2012-09-28T10:23:09+08:00       fluent.trace    {"message":"closed fluent socket object_id=242397940"}


至此已經完全實現2臺服務器上2個fluentd的橋接。

發佈了42 篇原創文章 · 獲贊 7 · 訪問量 22萬+
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章