詳述Linux系統中Apache配置與應用（二）

日誌分割

• 隨着網站的訪問量增加，默認情況下Apache的單個日誌文件也會越來越大

  • 日誌文件佔用磁盤空間很大
  • 查看相關信息不方便
• 對日誌文件進行分割
  • Apache自帶rotatelogs分割工具實現
  • 第三方工具cronolog分割

rotatelogs分割

• 配置網站的日誌文件轉交給rotatelogs分割處理
• 配置格式爲

ErrorLog 4| rotatelogs命令的絕對路徑-I日誌文件路徑/網站名-error %Y%m%d.log 86400"-
CustomLog "| rotatelogs命令路徑-I日誌文件路徑/網站名-access %Y%m%d.log 86400" combined

配合實例

[root@localhost ~]# yum install httpd -y    //安裝http服務
已加載插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: centos.ustc.edu.cn
 * extras: centos.ustc.edu.cn
...//省略部分內容...
已安裝:
  httpd.x86_64 0:2.4.6-90.el7.centos   
...//省略部分內容...
完畢！
[root@localhost bin]# cd /usr/sbin/    //進入sbin目錄
[root@localhost sbin]# ls rotat*      //查看目錄下是否生成命令
rotatelogs                         //安裝http服務後生成rotatelogs命令
[root@localhost sbin]# cd /var/log/httpd/    //進入http服務日誌存放目錄
[root@localhost httpd]# ls                  //查看是否生成日誌文件
[root@localhost httpd]#                   //安裝完成後並沒有生成目錄文件
[root@localhost httpd]# vim /etc/httpd/conf/httpd.conf  //進入http配置文件，進行常規配置
...//省略部分內容...
# prevent Apache from glomming onto all bound IP addresses.
#
Listen 192.168.144.133:80             //修改IPv4爲本地地址
#Listen 80                            //註釋掉IPv6監聽
...//省略部分內容...
# If your host doesn't have a registered DNS name, enter its IP address here.
#
ServerName www.kgc.com:80             //開啓域名服務

#
# Deny access to the entirety of your server's filesystem. You must
...//省略部分內容...
:wq
[root@localhost httpd]# systemctl stop firewalld.service     //關閉防火牆
[root@localhost httpd]# setenforce 0               //關閉增強型安全功能
[root@localhost httpd]# systemctl start httpd.service   //啓動http服務
[root@localhost httpd]# ls         //查看是否生成日誌文件
access_log  error_log               //服務開啓後成功生成日誌文件
[root@localhost httpd]# vim conf/httpd.conf   //進入更改主配置文件，做日誌分割
...//省略部分內容...
# container, that host's errors will be logged there and not here.
#
#ErrorLog "logs/error_log"      //複製此條目，並將此條目在下方粘貼
ErrorLog "| /usr/sbin/rotatelogs -l logs/www.kgc.com.error_%Y%m%log 86400"
//更改條目，引用rotatelogs命令，創建分割日誌條目名稱，每86400秒（一天）創建一次文件
# LogLevel: Control the number of messages logged to the error_log.
...//省略部分內容...
    # (Combined Logfile Format) you can use the following directive.
    #
    #CustomLog "logs/access_log" combined //複製此條目，並將此條目在下方粘貼
    CustomLog "| /usr/sbin/rotatelogs -l logs/www.kgc.com.access_%Y%m%dlog 86400" combined
    //更改條目，引用rotatelogs命令，創建分割日誌條目名稱，每86400秒（一天）創建一次文件
</IfModule>

<IfModule alias_module>
...//省略部分內容...
:wq
[root@localhost httpd]# systemctl stop httpd.service    //停止http服務
[root@localhost httpd]# systemctl start httpd.service   //啓動http服務
[root@localhost httpd]# cd ..
[root@localhost log]# cd ~
[root@localhost ~]# cd /var/log/httpd/      //重新進入站點文件
[root@localhost httpd]# ls
access_log  error_log  www.kgc.com.error_20191105log      //顯示分割除來的日誌文件
[root@localhost httpd]# date             //查看時間
2019年 11月 05日 星期二 19:27:50 CST
[root@localhost httpd]# date -s 11/06/19   //將時間設置後一天
2019年 11月 06日 星期三 00:00:00 CST
[root@localhost httpd]# systemctl stop httpd.service      //關閉http服務
[root@localhost httpd]# systemctl start httpd.service     //啓動服務 
[root@localhost httpd]# cd ~            //回到家目錄
[root@localhost ~]# cd /var/log/httpd/  //重新今日日誌存放目錄
[root@localhost httpd]# ls        //查看
access_log  error_log  www.kgc.com.error_20191105log  www.kgc.com.error_20191106log 
//生成新的日誌文件

第三方工具cronolog

• 源碼編譯安裝cronolog工具 cronolog安裝包下載
• 配置網站日誌文件轉交給cronolog分割處理
• 配置格式爲:

ErrorLog "| cronolog命令的絕對路徑日誌文件路徑/網站名-error_ %Y%m%d.log'
CustomLog "| cronolog命令的絕對路徑|日誌文件路徑/網站名%Y%m%d.log" combined

配置實例

重新開啓一臺Linux虛擬機，重新安裝http服務

[root@localhost ~]# yum install httpd -y      //安裝服務
已加載插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: centos.ustc.edu.cn
...//省略部分內容...
已安裝:
  httpd.x86_64 0:2.4.6-90.el7.centos 
...//省略部分內容...
[root@localhost ~]# mount.cifs //192.168.100.10/lamp-c7 /mnt   //遠程掛載目錄
Password for root@//192.168.100.10/lamp-c7:  
[root@localhost ~]# cd /mnt/              //進入掛載目錄
[root@localhost mnt]# ls       //查看是否成功掛載
apr-1.6.2.tar.gz       cronolog-1.6.2-14.el7.x86_64.rpm  httpd-2.4.29.tar.bz2  mysql-5.6.26.tar.gz
apr-util-1.6.0.tar.gz  Discuz_X2.5_SC_UTF8.zip           LAMP-php5.6.txt       nginx-1.12.0.tar.gz
awstats-7.6.tar.gz     error.png                         miao.jpg              php-5.6.11.tar.bz2
[root@localhost mnt]# rpm -ivh cronolog-1.6.2-14.el7.x86_64.rpm         //安裝軟件包
警告：cronolog-1.6.2-14.el7.x86_64.rpm: 頭V3 RSA/SHA256 Signature, 密鑰 ID 352c64e5: NOKEY
準備中...                          ################################# [100%]
正在升級/安裝...
   1:cronolog-1.6.2-14.el7            ################################# [100%]
[root@localhost mnt]# cd /usr/sbin/           //進入目錄
[root@localhost sbin]# ls cronolog*           //查看是否生成cronolog命令
cronolog                                      //成功安裝
[root@localhost sbin]# vim /etc/httpd/conf/httpd.conf    //進入編輯配置文件
...//省略部分內容...
# prevent Apache from glomming onto all bound IP addresses.
#
Listen 192.168.144.133:80          //設置監聽地址
#Listen 80

#
# Dynamic Shared Object (DSO) Support
...//省略部分內容...
# If your host doesn't have a registered DNS name, enter its IP address here.
#
ServerName www.kgc.com:80         //設置域名

#
# Deny access to the entirety of your server's filesystem. You must
...//省略部分內容...
# container, that host's errors will be logged there and not here.
#
ErrorLog "| /usr/sbin/cronolog logs/www.kgc.com.error_%Y%m%dlog"
//調用cronolog命令創建錯誤日誌文件
#
# LogLevel: Control the number of messages logged to the error_log.
...//省略部分內容...
   # (Combined Logfile Format) you can use the following directive.
    #
    CustomLog "| /usr/sbin/cronolog logs/www.kgc.com.access_%Y%m%dlog" combined
    //調用cronolog命令創建訪問日誌文件
</IfModule>
...//省略部分內容...
:wq
[root@localhost sbin]# systemctl stop firewalld.service    //關閉防火牆
[root@localhost sbin]# setenforce 0                        //關閉增強性安全功能
[root@localhost sbin]# systemctl start httpd.service       //開啓http服務
[root@localhost sbin]# cd /var/log/httpd/               //進入日誌文件目錄
[root@localhost httpd]# ls                             //查看
www.kgc.com.error_20191105log                           //生成日誌分割文件
[root@localhost httpd]# date                        //查看當前時間
2019年 11月 05日 星期二 20:11:57 CST
[root@localhost httpd]# date -s 11/06/19            //將時間設置爲後一天
2019年 11月 06日 星期三 00:00:00 CST
[root@localhost httpd]# systemctl stop httpd.service        //停止http服務
[root@localhost httpd]# systemctl start httpd.service       //啓動http服務
[root@localhost httpd]# cd ~                                 //回到家目錄
[root@localhost ~]# cd /var/log/httpd/                     //重新進入日誌目錄
[root@localhost httpd]# ls                                   //查看
www.kgc.com.error_20191105log  www.kgc.com.error_20191106log  //生成新的日誌文件

AWStats日誌分析系統介紹

AWStats日誌分析系統

• Perl語言開發的一款開源日誌分析系統
• 可用來分析Apache、Samba、 Vsftpd、 IIS等服務 器的訪問日誌
• 信息結合crond等計劃任務服務，可對日誌內容定期進行分析

部署AWStats日誌分析系統

• 重新開啓一臺Linux虛擬機，安裝http、DNS服務，並進行常規配置

[root@localhost ~]# yum install bind httpd -y
已加載插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirrors.163.com
...//省略部分內容...
已安裝:
  bind.x86_64 32:9.11.4-9.P2.el7                    httpd.x86_64 0:2.4.6-90.el7.centos       
...//省略部分內容...
[root@localhost ~]# vim /etc/named.conf 
...//省略部分內容...
options {
        listen-on port 53 { any; };
        listen-on-v6 port 53 { ::1; };
        directory       "/var/named";
        dump-file       "/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";
        memstatistics-file "/var/named/data/named_mem_stats.txt";
        recursing-file  "/var/named/data/named.recursing";
        secroots-file   "/var/named/data/named.secroots";
        allow-query     { any; };
...//省略部分內容...
[root@localhost ~]# vim /etc/named.rfc1912.zones
...//省略部分內容...
zone "kgc.com" IN {
        type master;
        file "kgc.com.zone";
        allow-update { none; };
};
...//省略部分內容...
[root@localhost named]# cp -p named.localhost kgc.com.zone
[root@localhost named]# vim kgc.com.zone 
$TTL 1D
@       IN SOA  @ rname.invalid. (
                                        0       ; serial
                                        1D      ; refresh
                                        1H      ; retry
                                        1W      ; expire
                                        3H )    ; minimum
        NS      @
        A       127.0.0.1
www IN  A       192.168.144.133
:wq
[root@localhost named]# vim /etc/httpd/conf/httpd.conf
...//省略部分內容...
# prevent Apache from glomming onto all bound IP addresses.
#
Listen 192.168.144.133:80
#Listen 80

#
# Dynamic Shared Object (DSO) Support
...//省略部分內容...
# If your host doesn't have a registered DNS name, enter its IP address here.
#
ServerName www.kgc.com:80

#
# Deny access to the entirety of your server's filesystem. You must
...//省略部分內容...
:wq
[root@localhost named]# systemctl stop firewalld.service 
[root@localhost named]# setenforce 0
[root@localhost named]# systemctl start named
[root@localhost named]# systemctl start httpd.service 

• 在客戶機中測試網站是否可以訪問
  
• 安裝AWStats軟件包

[root@localhost named]# cd /etc/httpd/                
[root@localhost httpd]# mount.cifs //192.168.100.10/lamp-c7 /mnt     //將軟件存放目錄掛載在mnt目錄下
Password for root@//192.168.100.10/lamp-c7:                           //直接回車
[root@localhost httpd]# cd /mnt/                                    //查看掛目錄
[root@localhost mnt]# ls
apr-1.6.2.tar.gz       cronolog-1.6.2-14.el7.x86_64.rpm  httpd-2.4.29.tar.bz2  mysql-5.6.26.tar.gz
apr-util-1.6.0.tar.gz  Discuz_X2.5_SC_UTF8.zip           LAMP-php5.6.txt       nginx-1.12.0.tar.gz
awstats-7.6.tar.gz     error.png                         miao.jpg              php-5.6.11.tar.bz2
[root@localhost mnt]# tar zxvf awstats-7.6.tar.gz -C /opt/                //解壓awstats工具在opt目錄
awstats-7.6/ 
awstats-7.6/tools/
awstats-7.6/tools/awstats_buildstaticpages.pl
...//省略部分內容...
[root@localhost mnt]# cd /opt/                                    //進入opt目錄                    
[root@localhost opt]# ls
awstats-7.6  rh
[root@localhost opt]# mv awstats-7.6/ /usr/local/awstats          //將解壓的文件移動至/usr/local/目錄下
[root@localhost opt]# cd /usr/local/                           //進入local目錄
[root@localhost local]# ls
awstats  bin  etc  games  include  lib  lib64  libexec  sbin  share  src
[root@localhost local]# cd awstats/                //進入awstats目錄
[root@localhost awstats]# ls
docs  README.md  tools  wwwroot
[root@localhost awstats]# cd tools/               //進入工具存放目錄
[root@localhost tools]# ls                   //查看可執行工具
awstats_buildstaticpages.pl  awstats_updateall.pl  httpd_conf          nginx               xslt
awstats_configure.pl         dolibarr              logresolvemerge.pl  urlaliasbuilder.pl
awstats_exportlib.pl         geoip_generator.pl    maillogconvert.pl   webmin

• 爲要統計的站點建立配置文件

[root@localhost tools]# ./awstats_configure.pl      //執行配置工具

----- AWStats awstats_configure 1.0 (build 20140126) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
...//省略部分內容...
Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
> /etc/httpd/conf/httpd.conf                              //輸入http配置文件路徑
...//省略部分內容...
-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y               //輸入y確認

-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> www.kgc.com                                          //輸入網頁域名
-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats                                          
Directory path to store config file(s) (Enter for default):
>                                                   //直接回車，默認存放配置文件路徑即可
...//省略部分內容...
Or if you have several config files and prefer having only one command:
/usr/local/awstats/tools/awstats_updateall.pl now
Press ENTER to continue... 

A SIMPLE config file has been created: /etc/awstats/awstats.www.kgc.com.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for 'www.kgc.com' with command:
> perl awstats.pl -update -config=www.kgc.com
You can also read your statistics for 'www.kgc.com' with URL:
> http://localhost/awstats/awstats.pl?config=www.kgc.com             //給出分析訪問網址

Press ENTER to finish...

[root@localhost tools]# cd /etc/httpd/conf               //進入http配置文件目錄
[root@localhost conf]# vim httpd.conf                    //編輯配置文件
...//省略部分內容...
# This is to permit URL access to scripts/files in AWStats directory.
#
<Directory "/usr/local/awstats/wwwroot">                //自動生成配置文件
    Options None
    AllowOverride None                          
#    Order allow,deny              //註釋掉訪問權限設置
#    Allow from all
     Require all granted          //添加條目分爲所有人都可以訪問
</Directory>
:wq

• 修改站點統計配置文件

[root@localhost etc]# cd /etc/awstats/        //進入配置awstats配置文件目錄
[root@localhost awstats]# ls                  //查看
awstats.www.kgc.com.conf                      //awstats配置文件
[root@localhost awstats]# ls /var/log/httpd/  //查看http服務日誌文件名稱
access_log  error_log
[root@localhost awstats]# vim awstats.www.kgc.com.conf     //進入awstats配置文件
...//省略部分內容...
# Example: "/pathtotools/logresolvemerge.pl *.log |"  
#
LogFile="/var/log/httpd/access_log"                         //找到次條目，更改要分析的日誌文件名稱即可

# Enter the log file type you want to analyze.
...//省略部分內容...
# Example: "C:/awstats_data_dir"
# Default: "."          (means same directory as awstats.pl)
#
DirData="/var/lib/awstats"                          //找到此條目，查看分析後數據的存放位置

# Relative or absolute web URL of your awstats cgi-bin directory.
...//省略部分內容...
:wq
[root@localhost awstats]# cd /var/lib/        //進入lib目錄
[root@localhost lib]# mkdir awstats            //創建awstats分析數據存放目錄
[root@localhost lib]# ls                       //查看目錄是否成功創建
AccountsService  chrony    flatpak   initramfs   misc    PackageKit  rpm-state       systemd  yum
alsa             color     fprint    ipa-client  mlocate   plymouth    rsyslog         tpm
alternatives     colord    games     iscsi       net-snmp      polkit-1    samba           tuned
authconfig       dav       gdm       libvirt     NetworkManager  postfix     selinux     udisks2
awstats          dbus      geoclue   lldpad      nfs          pulse      setroubleshoot  upower
bluetooth        dhclient  gssproxy  logrotate   ntp           rpcbind     sss             vmware
certmonger       dnsmasq   hyperv    machines    os-prober       rpm         stateless       xkb
[root@localhost lib]# systemctl restart httpd.service    //重啓http服務

• 在客戶機中測試awstats工具
  

• 這個時候訪問的awstats是沒有做統計分析的，我們需要執行更新數據工具來執行統計

[root@localhost lib]# cd /usr/local/awstats/  //進入awstats工具目錄
[root@localhost awstats]# ls
docs  README.md  tools  wwwroot
[root@localhost awstats]# cd tools/          //進入工具目錄
[root@localhost tools]# ls
awstats_buildstaticpages.pl  awstats_updateall.pl  httpd_conf          nginx               xslt
awstats_configure.pl         dolibarr              logresolvemerge.pl  urlaliasbuilder.pl
awstats_exportlib.pl         geoip_generator.pl    maillogconvert.pl   webmin
[root@localhost tools]# ./awstats_updateall.pl now   //執行工具進行更新，注意不要忘記後面加now
Running '"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=www.kgc.com -configdir="/etc/awstats"' to update config www.kgc.com
Create/Update database for config "/etc/awstats/awstats.www.kgc.com.conf" by AWStats version 7.6 (build 20161204)
From data in log file "/var/log/httpd/access_log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 48
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 0 corrupted records,                    //成功更新
 Found 0 old records,
 Found 48 new qualified records.

• 在客戶機重新訪問awstats

• 設置週期性計劃任務，自動統計訪問信息，並優化訪問路徑

[root@localhost tools]# crontab -e      //設置週期性計劃任務
*/5 * * * *     /usr/local/awstats/tools/./awstats_updateall.pl now   //設置每5分鐘執行一次
~
~
:wq
[root@localhost tools]# systemctl start crond               //啓動週期性計劃任務
[root@localhost tools]# cd /var/www/html/                   //進入http站點目錄
[root@localhost html]# vim aws.html                         //編輯網頁
<html>
  <head>    <meta http-equiv=refresh content="0;url=http://www.kgc.com/awstats/awstats.pl?config=www.kgc.com">                    //編輯網頁內容
   </head>
   <body></body>
</html>
:wq                        //保存退出

• 在客戶機中測試優化的訪問路徑是否可以使用