[2]supervisor管理:實現對異常中斷子進程的自動重啓(以nginx爲例訪問session)

參照建立的supervisor_tomcat服務：http://blog.51cto.com/sf1314/2128372 安裝supervisor管理看這博文

下面建立supervisor_nginx服務

       ①②③④⑤⑥⑦⑧⑨⑩

(1) YUM安裝Nginx (後續使用此種安裝，通過supervisor管理nginx)

centos7系統庫中默認是沒有nginx的rpm包的，所以我們自己需要先更新下rpm依賴庫

  ①使用yum安裝nginx需要包括Nginx的庫，安裝Nginx的庫

#rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm

  ②使用下面命令安裝nginx

#yum install -y nginx

  ③啓動Nginx

#service nginx start

或

#systemctl start nginx.service

  ④查看Nginx版本

# nginx -V

nginx version: nginx/1.14.0

built by gcc 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) 

built with OpenSSL 1.0.2k-fips  26 Jan 2017

TLS SNI support enabled

configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie'

（2）配置Nginx對tomcat的負載均衡

   ①查看tomcat後端的Connector port

# netstat -antup
Active Internet connections (servers and established)
Proto Recv-Q Send-Q  Local Address    Foreign Address   State       PID/Program name    
tcp   0      0   0.0.0.0:22       0.0.0.0:*      LISTEN      960/sshd            
tcp   0      0   0.0.0.0:5000     0.0.0.0:*       LISTEN      1350/nginx: master  
tcp   0      0   10.0.0.10:9001    0.0.0.0:*      LISTEN      1260/python         
tcp   0      96  10.0.0.10:22     10.0.0.1:62362   ESTABLISHED    1234/sshd: root@pts 
tcp6  0      0  :::8080          :::*         LISTEN      1262/java           
tcp6  0      0  :::8081          :::*         LISTEN      1273/java           
tcp6  0      0  :::22            :::*        LISTEN      960/sshd            
tcp6  0      0  127.0.0.1:8005      :::*        LISTEN      1262/java           
tcp6  0      0  127.0.0.1:8006      :::*        LISTEN      1273/java           
tcp6  0      0  :::8009           :::*        LISTEN      1262/java

[root@docker1 nginx]# curl 10.0.0.10:8080 確保原來的tomcat都能夠訪問

[root@docker1 nginx]# curl 10.0.0.10:8081

 ② 配置nginx負載均衡

採用tomcat部署，考慮到單個tomcat的最大也就能承受500左右的在線人數，這次採用了一個小的集羣部署，使用了2個tomcat，反向代理使用的nginx。

# cat /etc/nginx/nginx.conf
user  nginx;
worker_processes  5;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
events {
        worker_connections  1024;
}
http {
        include       mime.types;
        default_type  application/octet-stream;
        log_format  main  '$remote_addr - $remote_user [$time_local]    "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';
        access_log  /var/log/nginx/access.log  main;
        sendfile        on;
        #tcp_nopush     on;
        #keepalive_timeout  0;
        keepalive_timeout  65;
        #gzip  on;
        upstream  localhost   {                #這裏是localhost
          server   localhost:8080 weight=1;  
          server   localhost:8081 weight=2;  
        }  
        server {
        listen       80;
        server_name  localhost;
        #charset koi8-r;
        #access_log  logs/host.access.log  main;
        location / {
                root   html;
                index  index.html index.htm;
                        proxy_pass        http://localhost;  
                proxy_set_header  X-Real-IP  $remote_addr;  
                client_max_body_size  100m;  
        }
        #error_page  404              /404.html;
        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
                root   html;
        }
        }
}

  (3) 配置supervisor對Nginx的監控

  ① 編寫supervisor對Nginx的監控腳本

supervisor 監控nginx ，寫好配置文件之後，發現一直在重啓，排查之後發現是命令不對：

command = /usr/local/bin/nginx 這個命令默認是後臺啓動，但是supervisor不能監控後臺程序，所以supervisor就一直執行這個命令。

加上-g 'daemon off;'這個參數可解決這問題，這個參數的意思是在前臺運行。

command = /usr/local/bin/nginx  -g 'daemon off;'

完整的supervisor 監控nginx 配置如下：

# cat /etc/supervisor/config.d/nginx.ini 
[program:nginx]
command=/usr/sbin/nginx  -g 'daemon off;'
autostart=true
startsecs=10
autorestart=true
startretries=3
user=root
priority=998
;redirect_stderr=true      ;把stferr重定向到stdout，默認爲false，爲true的話，stderr的log會併入stdout的log
stdout_logfile=/var/log/nginx/nginx_stdout.log
stdout_logfile_maxbytes=10MB
stdout_logfile_backups=20
stderr_logfile=/var/log/nginx/nginx_stderr.log
stderr_logfile_maxbytes=10MB
stderr_logfile_backups=20

 ②測試是否配置成功

# supervisorctl -c /etc/supervisor/supervisord.conf 

nginx                            RUNNING   pid 1544, uptime 0:00:10

supervisor_tomcat1               RUNNING   pid 1545, uptime 0:00:10

supervisor_tomcat2               RUNNING   pid 1546, uptime 0:00:10

supervisor> exit

   ③ supervisor安裝後可能遇到的問題

問題1：

# supervisord -c /etc/supervisor/supervisord.conf 

Error: The directory named as part of the path /var/run/supervisor/supervisord.log does not exist.

For help, use /usr/bin/supervisord -h

# mkdir /var/run/supervisor/

# supervisord -c /etc/supervisor/supervisord.conf

問題2：

[root@docker1 ~]# supervisorctl 

http://localhost:9001 refused connection

supervisor> exit

# supervisorctl -c /etc/supervisor/supervisord.conf 

nginx                            FATAL     Exited too quickly (process log may have details)

supervisor_tomcat1               RUNNING   pid 1262, uptime 0:00:12

supervisor_tomcat2               RUNNING   pid 1264, uptime 0:00:12

supervisor> exit

問題3：

FATAL     Exited too quickly (process log may have details)    ;中文導致的

/etc/supervisor/conf.d/nginx.ini配置文件中，去除;的中文註釋

（4）NGINX反向代理tomcat

  ① 在三個tomcat的webapps/ROOT目錄下，分別添加session.jsp (注:每個tomcat下的標示不同)

[root@docker1 ROOT]# pwd
/usr/local/apache-tomcat-8.0.27/webapps/ROOT
[root@docker1 ROOT]# cat session.jsp 
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>shared session</title>
</head>
<body>
        <br>session id=<%=session.getId()%>
        <br>tomcat 1
</body>
</html>

-------------------------------------------------------------------------------

[root@docker1 ROOT]# cd /usr/local/apache-tomcat-8.0.28/webapps/ROOT/
[root@docker1 ROOT]# cat session.jsp 
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>shared session</title>
</head>
<body>
        <br>session id=<%=session.getId()%>
        <br>tomcat 2
</body>
</html>

 ② 訪問10.0.0.10/session.jsp是否分流到不同的tomcat (後續通過redis或memeched或mongodb要實現會話保持session)

從截圖中，可以看出，分別訪問了不同的tomcat，但是得到的session卻是相同的，說明達到了集羣的目的。

# pwd
/usr/local/apache-tomcat-8.0.27/conf
# ls                   #刪去了context.xml文件，context.xml連接redis
Catalina         catalina.properties      dump.rdb            server.xml        tomcat-users.xsd
catalina.policy  context.xml.20180712bak  logging.properties  tomcat-users.xml  web.xml
# cd ../webapps/ROOT/
# mv session.jsp index.jsp    #改名
# ls
index.jsp

對apache-tomcat-8.0.28也作此操作。

-----------------------------------------------------------------------------------

附：

NGINX YUM源nginx-release-centos-7-0.el7.ngx.noarch    http://down.51cto.com/data/2387451 

編譯安裝nginx

• 從官網下載Nginx

一、準備工作：

1、安裝必備工具：

# yum -y install gcc gcc-c++ autoconf automake

# yum -y install zlib zlib-devel openssl openssl-devel pcre-devel

說明：

pcre: 用來作地址重寫的功能。

zlib：nginx 的gzip模塊,傳輸數據打包，省流量（但消耗資源）。

openssl：提供ssl加密協議。

2、安裝之前，最好檢查一下是否已經安裝有nginx

# find -name nginx 

如果系統已經安裝了nginx，那麼就先卸載,直接把安裝目錄刪除就行                                                  

# yum remove nginx  此種刪除方式有風險，會刪去依賴包，建議如果刪除包rpm -e xxx --nodeps

如果系統已經安裝了nginx，那麼就先卸載                                                                                                                      

二、Nginx編譯安裝：

1、下載Nginx：http://nginx.org/en/download.html

# cd /usr/local

# wget http://nginx.org/download/nginx-1.13.0.tar.gz 

2、解壓編譯：

# tar -zxvf nginx-1.13.0.tar.gz 

# cd nginx-1.13.0/

# ./configure \

--prefix=/etc/nginx \

--sbin-path=/usr/sbin/nginx \

--conf-path=/etc/nginx/nginx.conf \

--error-log-path=/var/log/nginx/error.log \

--http-log-path=/var/log/nginx/access.log \

--pid-path=/var/run/nginx.pid \

--lock-path=/var/run/nginx.lock \

--http-client-body-temp-path=/var/cache/nginx/client_temp \

--http-proxy-temp-path=/var/cache/nginx/proxy_temp \

--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \

--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \

--http-scgi-temp-path=/var/cache/nginx/scgi_temp \

--user=nginx \

--group=nginx \

--with-http_ssl_module \

--with-http_realip_module \

--with-http_addition_module \

--with-http_sub_module \

--with-http_dav_module \

--with-http_flv_module \

--with-http_mp4_module \

--with-http_gunzip_module \

--with-http_gzip_static_module \

--with-http_random_index_module \

--with-http_secure_link_module \

--with-http_stub_status_module \

--with-http_auth_request_module \

--with-mail \

--with-mail_ssl_module \

--with-file-aio \

--with-ipv6 \

--with-http_spdy_module \

--with-cc-opt='-O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic'

上面的參數的作用可以通過--help來查看下文附加1中有提供

$ ./configure --help

編譯日誌：

checking for OS

 + Linux 3.10.0-229.4.2.el7.x86_64 x86_64

checking for C compiler ... found

 + using GNU C compiler

 + gcc version: 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) 

checking for gcc -pipe switch ... found

checking for gcc builtin atomic operations ... found

checking for C99 variadic macros ... found

checking for gcc variadic macros ... found

checking for unistd.h ... found

checking for inttypes.h ... found

checking for limits.h ... found

checking for sys/filio.h ... not found

checking for sys/param.h ... found

checking for sys/mount.h ... found

checking for sys/statvfs.h ... found

checking for crypt.h ... found

checking for Linux specific features

checking for epoll ... found

checking for EPOLLRDHUP ... found

checking for O_PATH ... found

checking for sendfile() ... found

checking for sendfile64() ... found

checking for sys/prctl.h ... found

checking for prctl(PR_SET_DUMPABLE) ... found

checking for sched_setaffinity() ... found

checking for crypt_r() ... found

checking for sys/vfs.h ... found

checking for poll() ... found

checking for /dev/poll ... not found

checking for kqueue ... not found

checking for crypt() ... not found

checking for crypt() in libcrypt ... found

checking for F_READAHEAD ... not found

checking for posix_fadvise() ... found

checking for O_DIRECT ... found

checking for F_NOCACHE ... not found

checking for directio() ... not found

checking for statfs() ... found

checking for statvfs() ... found

checking for dlopen() ... not found

checking for dlopen() in libdl ... found

checking for sched_yield() ... found

checking for SO_SETFIB ... not found

checking for SO_REUSEPORT ... found

checking for SO_ACCEPTFILTER ... not found

checking for TCP_DEFER_ACCEPT ... found

checking for TCP_KEEPIDLE ... found

checking for TCP_FASTOPEN ... not found

checking for TCP_INFO ... found

checking for accept4() ... found

checking for eventfd() ... found

checking for int size ... 4 bytes

checking for long size ... 8 bytes

checking for long long size ... 8 bytes

checking for void * size ... 8 bytes

checking for uint64_t ... found

checking for sig_atomic_t ... found

checking for sig_atomic_t size ... 4 bytes

checking for socklen_t ... found

checking for in_addr_t ... found

checking for in_port_t ... found

checking for rlim_t ... found

checking for uintptr_t ... uintptr_t found

checking for system byte ordering ... little endian

checking for size_t size ... 8 bytes

checking for off_t size ... 8 bytes

checking for time_t size ... 8 bytes

checking for setproctitle() ... not found

checking for pread() ... found

checking for pwrite() ... found

checking for sys_nerr ... found

checking for localtime_r() ... found

checking for posix_memalign() ... found

checking for memalign() ... found

checking for mmap(MAP_ANON|MAP_SHARED) ... found

checking for mmap("/dev/zero", MAP_SHARED) ... found

checking for System V shared memory ... found

checking for POSIX semaphores ... not found

checking for POSIX semaphores in libpthread ... found

checking for struct msghdr.msg_control ... found

checking for ioctl(FIONBIO) ... found

checking for struct tm.tm_gmtoff ... found

checking for struct dirent.d_namlen ... not found

checking for struct dirent.d_type ... found

checking for sysconf(_SC_NPROCESSORS_ONLN) ... found

checking for openat(), fstatat() ... found

checking for getaddrinfo() ... found

checking for PCRE library ... found

checking for PCRE JIT support ... found

checking for OpenSSL library ... found

checking for zlib library ... found

creating objs/Makefile

 

Configuration summary

  + using system PCRE library

  + using system OpenSSL library

  + md5: using OpenSSL library

  + sha1: using OpenSSL library

  + using system zlib library

 

  nginx path prefix: "/usr"

  nginx binary file: "/usr/sbin/nginx"

  nginx configuration prefix: "/etc/nginx"

  nginx configuration file: "/etc/nginx/nginx.conf"

  nginx pid file: "/var/run/nginx/nginx.pid"

  nginx error log file: "/var/log/nginx/error.log"

  nginx http access log file: "/var/log/nginx/http.log"

  nginx http client request body temporary files: "/var/tmp/nginx/client"

  nginx http proxy temporary files: "/var/tmp/nginx/proxy"

  nginx http fastcgi temporary files: "/var/tmp/nginx/fcgi"

  nginx http uwsgi temporary files: "uwsgi_temp"

  nginx http scgi temporary files: "scgi_temp"

好像很成功。

3、安裝:

# make && make install

4、啓動：

# /usr/local/nginx/sbin/nginx

5、其它：

停止：

# /usr/local/nginx/sbin/nginx -s stop 

查看版本及安裝的模塊(大寫的V)：

# /usr/local/nginx/sbin/nginx -V