本文分享自華爲雲社區《nginx.conf以configmap文件形式掛載到nginx容器中以及subpath使用場景》,作者:可以交個朋友。
背景
nginx.conf通過configmap文件形式掛載到容器內,可以更加方便的修改nginx.conf配置
方案簡介
將配置文件nginx.conf以configmap文件的方式掛載到容器中。爲了更通用,可以將使用主nginx.conf include 指定xx.conf方式,主nginx.conf作爲一個cm,具體xx.conf對應一個cm
將nginx.conf作爲configmap掛載到容器中
1.創建configmap
apiVersion: v1 kind: ConfigMap metadata: name: nginx-config namespace: default data: nginx.conf: |+ user nginx; worker_processes 8; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; keepalive_timeout 65; #gzip on; include /etc/nginx/conf.d/*.conf; } --- apiVersion: v1 kind: ConfigMap metadata: name: nginx-server-config namespace: default data: server1.conf: |+ server { listen 80; server_name server1.com; location / { root /usr/share/nginx/html/; index index.html index.htm; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } server2.conf: |+ server { listen 81; server_name server2.com; location / { root /usr/share/nginx/html/; index index.html index.htm; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } }
2.部署nginx業務使用對應的cm
apiVersion: apps/v1 kind: Deployment metadata: labels: version: v1 name: test-reload namespace: default spec: progressDeadlineSeconds: 600 replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: app: test-reload template: metadata: labels: app: test-reload spec: containers: - image: nginx:latest imagePullPolicy: Always name: container-1 volumeMounts: - mountPath: /etc/nginx/conf.d name: vol-168233491311961268 - mountPath: /etc/nginx/nginx.conf name: vol-168249948123126427 readOnly: true subPath: nginx.conf dnsPolicy: ClusterFirst imagePullSecrets: - name: default-secret restartPolicy: Always volumes: - configMap: defaultMode: 420 name: nginx-server-config name: vol-168233491311961268 - configMap: defaultMode: 420 name: nginx-config name: vol-168249948123126427
subpath拓展
subpath的作用如下:
- 避免覆蓋。如果掛載路徑是一個已存在的目錄,則目錄下的內容不會被覆蓋。直接將configMap/Secret掛載在容器的路徑,會覆蓋掉容器路徑下原有的文件,使用subpath選定configMap/Secret的指定的key-value掛載在容器中,則不會覆蓋掉原目錄下的其他文件
- 文件隔離。pod中含有多個容器公用一個日誌volume,不同容器日誌路徑掛載的到不同的子目錄,而不是根路徑(Subpath目錄會在底層存儲自動創建且權限爲777,無需手動創建)
避免覆蓋效果演示
1.創建一個工作負載nginx,並用普通方式掛載configmap配置文件
apiVersion: v1 kind: ConfigMap metadata: name: config data: test-subpath.conf: |+ test subpath; --- apiVersion: apps/v1 kind: Deployment metadata: labels: app: test name: test spec: replicas: 1 selector: matchLabels: app: test template: metadata: labels: app: test spec: volumes: - configMap: defaultMode: 420 name: config name: vol-168249948123126427 containers: - image: centos:latest name: centos command: - /bin/bash args: - -c - while true;do sleep 1 && echo hello;done volumeMounts: - mountPath: /tmp name: vol-168249948123126427
2.使用docker inspect ${容器id}命令查看容器掛載信息,掛載目標爲tmp目錄,tmp目錄下原有內容被覆蓋
[root@test-746c64649c-pzztn /]# ls -l /tmp/ total 0 lrwxrwxrwx 1 root root 24 Feb 27 03:02 test-subpath.conf -> ..data/test-subpath.conf
3.創建一個工作負載nginx,並用subpath方式掛載configmap配置文件
apiVersion: v1 kind: ConfigMap metadata: name: config data: test-subpath.conf: |+ test subpath; --- apiVersion: apps/v1 kind: Deployment metadata: labels: app: test name: test spec: replicas: 1 selector: matchLabels: app: test template: metadata: labels: app: test spec: volumes: - configMap: defaultMode: 420 name: config name: vol-168249948123126427 containers: - image: centos:latest name: centos command: - /bin/bash args: - -c - while true;do sleep 1 && echo hello;done volumeMounts: - mountPath: /tmp/test-subpath.conf name: vol-168249948123126427 subPath: test-subpath.conf
4.使用docker inspect ${容器Id}命令查看容器掛載信息,掛載目標爲test-subpath.conf文件,所以tmp目錄下原來的文件不會被覆蓋
[root@test-7b64fd6bb-56lpp /]# ls -l /tmp/ total 12 -rwx------ 1 root root 701 Dec 4 2020 ks-script-esd4my7v -rwx------ 1 root root 671 Dec 4 2020 ks-script-eusq_sc5 -rw-r--r-- 1 root root 14 Feb 27 03:07 test-subpath.conf
文件隔離演示
1.創建工作負載test,使用hostPath卷類型持久化日誌文件
apiVersion: apps/v1 kind: Deployment metadata: labels: app: test name: test spec: replicas: 2 selector: matchLabels: app: test template: metadata: labels: app: test spec: volumes: - hostPath: path: /tmp/log #該路徑必須在節點上已存在 name: vol-168249948123126427 containers: - image: centos:latest name: centos env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name command: - /bin/bash args: - -c - while true;do echo $(POD_NAME) >> /tmp/log/app.log && sleep 900 ;done volumeMounts: - mountPath: /tmp/log name: vol-168249948123126427 subPathExpr: $(POD_NAME)
2.兩個Pod實例調度至同一個節點
[root@test ~]# kubectl get pod -owide -l app=test NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES test-69dfc665cd-2nhg5 1/1 Running 0 95s 172.16.4.59 172.16.2.172 <none> <none> test-69dfc665cd-z7rsj 1/1 Running 0 77s 172.16.4.25 172.16.2.172 <none> <none>
3.進入容器內查看日誌文件
[root@test ~]# kubectl exec -it test-69dfc665cd-2nhg5 bash [root@test-69dfc665cd-2nhg5 /]# cat /tmp/log/app.log test-69dfc665cd-2nhg5 [root@test-69dfc665cd-2nhg5 /]# exit exit [root@test ~]# kubectl exec -it test-69dfc665cd-z7rsj bash [root@test-69dfc665cd-z7rsj /]# cat /tmp/log/app.log test-69dfc665cd-z7rsj
4.在節點上查看掛載路徑,每個Pod的日誌文件用目錄進行隔離,目錄名爲Pod名稱
[root@172 log]# pwd /tmp/log [root@172 log]# ll total 0 drwxr-xr-x 2 root root 60 Feb 27 15:08 test-69dfc665cd-2nhg5 drwxr-xr-x 2 root root 60 Feb 27 15:09 test-69dfc665cd-z7rsj [root@172 log]# cat test-69dfc665cd-2nhg5/app.log test-69dfc665cd-2nhg5 [root@172 log]# cat test-69dfc665cd-z7rsj/app.log test-69dfc665cd-z7rsj