******************理論部分*****************
前言：
在數據爲王的時代，日誌管理是一個繞不開的話題，相應的開源軟件有不少，比如熱門的三件套：Logstash、ElasticSearch、Kibana，雖然功能強大，但是配置複雜。相比較而言，rsyslog更容易快速上手。
Rsyslog:
rsyslog是一款自由軟件，GPL(General Public License)的lincesed增強的syslogd.功能強大，有開源web loganalyzer支持，同時可以將收集日誌存在mysql中，方便分析、審計。具體功能可以查看官網.
日誌：
歷史日誌
歷史事件：時間，事件
日誌級別：事件的關鍵性程度，loglevel.
系統日誌服務：
syslog:(CentOS 5.x)
守護進程：
syslogd: system 記錄系統日誌
klogd: kernel 記錄內核日誌
rsyslog:(CentOS6以後)
守護進程：
syslogd
klogd
注:rsyslog是syslog的下一代版本.
Rsyslog的特性:
a.多線程;
b.使用的協議:UDP, TCP, SSL, TLS, RELP
c.可用於實現日誌存儲的數據庫:MYSQL, PGSQL, ORACLE等;
d.強大的過濾器，可實現過濾日誌信息中任何部分;
日誌收集方：
facility: 設施，從功能或程序上對日誌進行分類：
auth, authrive, cron, daemon, kern, lpr, mail, mark, news, security, user, uucp, local0-local7, syslog
priority:
info, debug, notice, warning, error,crit(critical), alert, emerg(pamic)
指定級別：
*：所有級別
none：沒有級別
priority：此級別及更高級別的日誌信息;
facility.priority /var/log/message
程序環境：
主程序：rsyslogd
配置文件：/etc/rsyslog.conf
服務腳本：/etc/rc.d/init.d/rsyslog
rsyslog.conf
RULES:
facility.priority target
target:
文件路徑：記錄於指定的日誌文件中，通常應該在/var/log目錄下; 文件路徑前的“-”表示異步寫入;
用戶：將日誌通知給指定用戶
*：所有用戶;
日誌服務器：@host
host: 必須要監聽在tcp或udp協議514端口上提供服務;
管道； |COMMAND
文件記錄的日誌格式：
事件產生的日期時間 主機 進程(pid): 事件內容
有些日誌記錄二進制格式： /var/log/wtmp, /var/log/btmp
/var/log/wtmp: 當前系統上成功登陸的日誌;
last
/var/log/btmp:
lastb
lastlog命令: 顯示當前系統每一個用戶最近一次的登陸時間;
******************實操部分*****************
實驗要求：
a.配置簡單的rsyslog服務器，增加一臺客戶端機器，由rsyslog服務器收集客戶端生成的日誌信息；
b.配置rsyslog+loganalyzer+mysql組合型日誌服務器，要求客戶端生成的日誌，由rsyslog服務器收集，並由mysql數據庫記錄rsyslog服務器的日誌，最後由loganalyzer工具在前端界面展示出來.
實驗環境：
系統：CentOS 6.7 x3
主機名及服務器作用：
CentOS 6.7:
7-200: rsyslog Server
7-201: Client
7-202: Mysql
1.1 安裝rsyslog服務器：
[root@7-200 ~]# rpm -qa rsyslog rsyslog-5.8.10-10.el6_6.x86_64
1.2 打開rsyslog的指定模塊，監聽在指定套接字：
[root@7-200 ~]# vim /etc/rsyslog.conf ... 12 # Provides UDP syslog reception 13 $ModLoad imudp 14 $UDPServerRun 514 15 16 # Provides TCP syslog reception 17 $ModLoad imtcp 18 $InputTCPServerRun 514 ... //13-14行表示監聽UDP協議，打開對UDP協議的收集日誌的模塊的支持，讓它監聽在UDP協議的514端口. //17-18行表示監聽TCP協議，打開對TCP協議的收集日誌的模塊的支持，讓它監聽在TCP協議的514端口.
注：對於開啓UDP協議和TCP協議的選擇上，UDP更快，TCP在日誌信息記錄時更可靠,此處都打開.
1.3 啓動rsyslogd服務，查看監聽端口：
[root@7-200 ~]# service rsyslog start Starting system logger: [ OK ] 3059/rsyslogd [root@7-200 ~]# netstat -tunlp |grep rsyslogd tcp 0 0 0.0.0.0:514 0.0.0.0:* LISTEN 3059/rsyslogd tcp 0 0 :::514 :::* LISTEN 3059/rsyslogd udp 0 0 0.0.0.0:514 0.0.0.0:* 3059/rsyslogd udp 0 0 :::514 :::* 3059/rsyslogd [root@7-200 ~]#
//可以看到514端口都已監聽在TCP和UDP上,此時其他主機就可以往該日誌服務器發日誌了.
1.4 開啓Client服務器7-201，往日誌服務器7-200發送日誌：
[root@7-201 ~]# rpm -qa rsyslog rsyslog-5.8.10-10.el6_6.x86_64
1.5 配置rsyslog.conf:
[root@7-201 ~]# vim /etc/rsyslog.conf 42 #*.info;mail.none;authpriv.none;cron.none /var/log/messages 43 *.info;mail.none;authpriv.none;cron.none @10.68.7.200 //備份第43行，修改日誌服務器地址.
1.6 重啓客戶端日誌服務器程序：
[root@7-201 ~]# service rsyslog restart Shutting down system logger: [ OK ] Starting system logger: [ OK ] [root@7-201 ~]#
1.7 在7-201客戶端服務器安裝vsftp服務，然後在7-200服務器端/var/log/messages查看：
[root@7-201 ~]# yum -y install vsftpd;date ... Installed: vsftpd.x86_64 0:2.2.2-14.el6 Complete! Sun Aug 21 22:43:21 EDT 2016
[root@7-200 ~]# tail /var/log/messages Aug 21 22:43:21 7-201 yum[3187]: Installed: vsftpd-2.2.2-14.el6.x86_64
客戶端日誌在rsyslog服務端收集，保存在mysql 數據庫中，然後通過前端頁面展示工具展示出來：
2.1 在7-200服務端安裝rsyslog-mysql軟件：
[root@7-200 ~]# yum -y install rsyslog-mysql [root@7-200 ~]# rpm -ql rsyslog-mysql /lib64/rsyslog/ommysql.so /usr/share/doc/rsyslog-mysql-5.8.10 /usr/share/doc/rsyslog-mysql-5.8.10/createDB.sql //注意該文件. [root@7-200 ~]#
2.2 用二進制方式安裝mariadb服務器並完成相關操作:
[root@7-202 ~]# groupadd -r -g 306 mysql [root@7-202 ~]# useradd -r -g 306 -u 306 mysql [root@7-202 ~]# tar xf mariadb-5.5.46-linux-x86_64.tar.gz -C /usr/local [root@7-202 ~]# ln -sv /usr/local/mariadb-5.5.46-linux-x86_64/ /usr/local/mysql `/usr/local/mysql' -> `/usr/local/mariadb-5.5.46-linux-x86_64/' [root@7-202 ~]# cd /usr/local/mysql [root@7-202 mysql]# chown -R root:mysql ./* [root@7-202 mysql]# scripts/mysql_install_db --datadir=/mydata/data --user=mysql WARNING: The host '7-202' could not be looked up with resolveip. This probably means that your libc libraries are not 100 % compatible with this binary MariaDB version. The MariaDB daemon, mysqld, should work normally with the exception that host name resolving will not work. This means that you should use IP addresses instead of hostnames when specifying MariaDB privileges ! Installing MariaDB/MySQL system tables in '/mydata/data' ... 160824 4:11:59 [Note] ./bin/mysqld (mysqld 5.5.46-MariaDB) starting as process 2538 ... OK Filling help tables... 160824 4:12:01 [Note] ./bin/mysqld (mysqld 5.5.46-MariaDB) starting as process 2547 ... OK To start mysqld at boot time you have to copy support-files/mysql.server to the right place for your system PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER ! To do so, start the server, then issue the following commands: './bin/mysqladmin' -u root password 'new-password' './bin/mysqladmin' -u root -h 7-202 password 'new-password' Alternatively you can run: './bin/mysql_secure_installation' which will also give you the option of removing the test databases and anonymous user created by default. This is strongly recommended for production servers. See the MariaDB Knowledgebase at http://mariadb.com/kb or the MySQL manual for more instructions. You can start the MariaDB daemon with: cd '.' ; ./bin/mysqld_safe --datadir='/mydata/data' You can test the MariaDB daemon with mysql-test-run.pl cd './mysql-test' ; perl mysql-test-run.pl Please report any problems at http://mariadb.org/jira The latest information about MariaDB is available at http://mariadb.org/. You can find additional information about the MySQL part at: http://dev.mysql.com Support MariaDB development by buying support/new features from MariaDB Corporation Ab. You can contact us about this at [email protected]. Alternatively consider joining our community based development effort: http://mariadb.com/kb/en/contributing-to-the-mariadb-project/ [root@7-202 mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysqld [root@7-202 mysql]# chkconfig --add mysqld [root@7-202 mysql]# chkconfig --list mysqld mysqld 0:off 1:off 2:on 3:on 4:on 5:on 6:off [root@7-202 mysql]# [root@7-202 mysql]# mkdir /etc/mysql [root@7-202 mysql]# cp support-files/my-large.cnf /etc/mysql/my.cnf [root@7-202 mysql]# vim /etc/mysql/my.cnf ... 在thread_concurrency = 8 這一行下面添加如下三行 datadir = /mydata/data innodb_file_per_table = on skip_name_resolve = on ... [root@7-202 mysql]# service mysqld start Starting MySQL... SUCCESS! [root@7-202 mysql]# mysql Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.5.46-MariaDB-log MariaDB Server Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> mysql> GRANT all on Syslog.* to 'syslog'@'10.68.7.%' Identified by 'syslog'; Query OK, 0 rows affected (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec) mysql> exit Bye [root@7-202 mysql]# [root@7-202 mysql]# vim /etc/my.cnf ...添加如下兩行： skip_name_resolv = on innodb_file_per_table = on
2.3 從rsyslog服務器上遠程登錄mysql服務器：
[root@7-200 ~]# mysql -usyslog -h10.68.7.202 -psyslog Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 8 Server version: 5.5.46-MariaDB-log MariaDB Server Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql>
2.4 將軟件rsyslog-mysql生成的sql語句導入數據庫：
[root@7-200 ~]# rpm -ql rsyslog-mysql /lib64/rsyslog/ommysql.so /usr/share/doc/rsyslog-mysql-5.8.10 /usr/share/doc/rsyslog-mysql-5.8.10/createDB.sql [root@7-200 ~]# mysql -usyslog -h10.68.7.202 -p </usr/share/doc/rsyslog-mysql-5.8.10/createDB.sql Enter password: [root@7-200 ~]#
//此時，可以登錄到數據庫查看到Syslog庫及對應的表.
2.5 配置rsyslog能使用mysql服務器：
... 20 $ModLoad ommysql //增加該項，添加到MonLoad區域附近. 42 *.info;mail.none;authpriv.none;cron.none :ommysql:10.68.7.202,Syslog,syslog,syslog //在第42行處添加，注意右邊的格式
2.6 重啓rsyslog服務:
[root@7-200 ~]# service rsyslog restart Shutting down system logger: [ OK ] Starting system logger: [ OK ] [root@7-200 ~]#
2.7 然後在rsyslog服務器安裝vsftpd服務，Client 7-201服務器安裝samba服務，生成的日誌在mysql數據庫中查看:
[root@7-201 ~]# yum -y install samba ... Running Transaction Installing : samba-3.6.23-20.el6.x86_64 1/1 Verifying : samba-3.6.23-20.el6.x86_64 1/1 Installed: samba.x86_64 0:3.6.23-20.el6 Complete!
[root@7-200 ~]# yum -y install vsftpd ... Installed: vsftpd.x86_64 0:2.2.2-14.el6 Complete! [root@7-200 ~]# mysql -usyslog -h10.68.7.202 -psyslog Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 16 Server version: 5.5.46-MariaDB-log MariaDB Server Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> use Syslog; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +------------------------+ | Tables_in_Syslog | +------------------------+ | SystemEvents | | SystemEventsProperties | +------------------------+ 2 rows in set (0.00 sec) mysql> select * from SystemEvents\G; *************************** 1. row *************************** ID: 1 CustomerID: NULL ReceivedAt: 2016-08-24 04:26:53 DeviceReportedTime: 2016-08-24 04:26:53 Facility: 0 Priority: 6 FromHost: 7-200 Message: imklog 5.8.10, log source = /proc/kmsg started. NTSeverity: NULL Importance: NULL EventSource: NULL EventUser: NULL EventCategory: NULL EventID: NULL EventBinaryData: NULL MaxAvailable: NULL CurrUsage: NULL MinUsage: NULL MaxUsage: NULL InfoUnitID: 1 SysLogTag: kernel: EventLogType: NULL GenericFileName: NULL SystemID: NULL *************************** 2. row *************************** ID: 2 CustomerID: NULL ReceivedAt: 2016-08-24 04:26:53 DeviceReportedTime: 2016-08-24 04:26:53 Facility: 5 Priority: 6 FromHost: 7-200 Message: [origin software="rsyslogd" swVersion="5.8.10" x-pid="2367" x-info="http://www.rsyslog.com"] start NTSeverity: NULL Importance: NULL EventSource: NULL EventUser: NULL EventCategory: NULL EventID: NULL EventBinaryData: NULL MaxAvailable: NULL CurrUsage: NULL MinUsage: NULL MaxUsage: NULL InfoUnitID: 1 SysLogTag: rsyslogd: EventLogType: NULL GenericFileName: NULL SystemID: NULL *************************** 3. row *************************** ID: 3 CustomerID: NULL ReceivedAt: 2016-08-24 04:27:27 DeviceReportedTime: 2016-08-24 04:27:27 Facility: 1 Priority: 6 FromHost: 7-200 Message: Installed: vsftpd-2.2.2-14.el6.x86_64 NTSeverity: NULL Importance: NULL EventSource: NULL EventUser: NULL EventCategory: NULL EventID: NULL EventBinaryData: NULL MaxAvailable: NULL CurrUsage: NULL MinUsage: NULL MaxUsage: NULL InfoUnitID: 1 SysLogTag: yum[2397]: EventLogType: NULL GenericFileName: NULL SystemID: NULL *************************** 4. row *************************** ID: 4 CustomerID: NULL ReceivedAt: 2016-08-24 04:30:03 DeviceReportedTime: 2016-08-24 04:30:03 Facility: 1 Priority: 6 FromHost: 7-201 Message: Installed: samba-3.6.23-20.el6.x86_64 NTSeverity: NULL Importance: NULL EventSource: NULL EventUser: NULL EventCategory: NULL EventID: NULL EventBinaryData: NULL MaxAvailable: NULL CurrUsage: NULL MinUsage: NULL MaxUsage: NULL InfoUnitID: 1 SysLogTag: yum[4787]: EventLogType: NULL GenericFileName: NULL SystemID: NULL 4 rows in set (0.00 sec) ERROR: No query specified mysql> //可以看到7-200主機安裝的vsftpd服務日誌和7-201主機安裝的samba服務日誌已經記錄在mysql數據庫中了.
3.1 配置rsyslog前端展示界面工具loganalyzer:
3.1.1 配置webserver,支持PHP
[root@7-200 ~]# yum install httpd php php-mysql php-gd [root@7-200 ~]# service httpd start [root@7-200 ~]# vim /etc/httpd/conf/httpd.conf ... 402 DirectoryIndex index.php index.html index.html.var //此處添加index.php文件，使httpd服務支持php.
3.1.2 [root@7-200 ~]# vim /var/www/html/index.php
//添加如下內容：
<?php
$conn=mysql_connect('10.68.7.202','syslog','syslog');
if ($conn)
echo "OK";
else
echo "Not OK";
phpinfo();
?>
:wq3.1.3 瀏覽器查看http與mysql、php的連接情況，即lamp環境：
//顯示OK則說明http與mysql連接正常.
3.2 安裝oganalyzer工具:
[root@7-200 ~]# tar -zxvf loganalyzer-3.6.5.tar.gz -C /var/www/html/ [root@7-200 ~]# cd /var/www/html/loganalyzer-3.6.5 [root@7-200 loganalyzer-3.6.5]# [root@7-200 loganalyzer-3.6.5]# mv src/ ../loganalyzer [root@7-200 loganalyzer-3.6.5]# cp contrib/*.sh ../loganalyzer [root@7-200 loganalyzer-3.6.5]# cd ../loganalyzer [root@7-200 loganalyzer]# ls *.sh configure.sh secure.sh [root@7-200 loganalyzer]# chmod +x *.sh [root@7-200 loganalyzer]# ./configure.sh [root@7-200 loganalyzer]# ./secure.sh [root@7-200 loganalyzer]# chmod 666 config.php [root@7-200 loganalyzer]# service httpd restart Stopping httpd: [ OK ] Starting httpd: httpd: apr_sockaddr_info_get() failed for 7-200 httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName [ OK ] [root@7-200 loganalyzer]#
3.3 此時就可以在瀏覽器訪問了，http://10.68.7.200/loganalyzer/install.php 第一次訪問時需加install.php,之後就不用加了.
一直下一步：
一直下一步：
至此，rsyslog+loganalyzer+lamp組合型日誌服務器安裝成功，至於loganalyzer工具的更多功能，此處不再詳述！