網上很多文檔都說deployment 配置readiness就可以實現無損rolling update,事實真的是這樣嗎?
最近我們在生產環境發現一個現象,當deployment 定義的 replicas 實例數只有1個的時候,執行rollingupdate 會有坑
按照官方文檔的說明,deployment 執行rollingupdate 在啓動時會先拉起新版本pod再幹掉舊版本的pod,逐步將所有pod 升級成新版本
但實際測試過程中發現,執行rollingupdate 時,舊replicas 中的pod 立馬會被幹掉一個,如所示:
rollingupdate 前:
root@ubuntu:~ # kubectl get rs NAME DESIRED CURRENT READY AGE webtest-static-test-com-56678f6856 1 1 1 50m
rollingupdate 中:
root@ubuntu:~ # kubectl get rs NAME DESIRED CURRENT READY AGE webtest-static-test-com-56678f6856 0 0 0 50m webtest-static-test-com-7d785c987 1 1 0 25m
執行rollingupdate 時,deployment 會創建一個新的rs,隨即將舊rs 中的pod 幹掉
可以看到這裏不管新舊pod READY 的字段都是0,這裏會有問題,執行rollingupdate 如果新版本服務啓動比較慢(例如tomcat),那這段時間服務都不可用
rollingupdate 後:
root@ubuntu:~ # kubectl get rs NAME DESIRED CURRENT READY AGE webtest-static-test-com-56678f6856 1 1 1 50m webtest-static-test-com-7d785c987 0 0 0 25m
從另外一個終端每隔1s 發起一次curl 請求,可以看到升級期間服務中斷:
root@ubuntu: ~ # for i in {0..99};do curl http://webtest-static.test.com/index.html ;echo;sleep 1;done This is server01 - Version - 2 This is server01 - Version - 2 <html> <head><title>503 Service Temporarily Unavailable</title></head> <body> <center><h1>503 Service Temporarily Unavailable</h1></center> <hr><center>nginx/1.15.8</center> </body> </html> <html> <head><title>503 Service Temporarily Unavailable</title></head> <body> <center><h1>503 Service Temporarily Unavailable</h1></center> <hr><center>nginx/1.15.8</center> </body> </html> ... ... This is server01 - Version - 3 This is server01 - Version - 3
當實例數(replicas 數) > 1 時,rollingupdate 過程中服務不會中斷
附:deployment yaml
apiVersion: extensions/v1beta1 kind: Deployment metadata: name: webtest-static-test-com spec: strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 1 replicas: 1 template: metadata: labels: app: webtest-static-test-com domain: webtest-static.test.com version: v1 spec: imagePullSecrets: - name: registry.cn-hangzhou.aliyuncs.com containers: - name: webtest-static-sysop-duowan-com image: registry.cn-hangzhou.aliyuncs.com/test/webtest_static:2.6 command: ["/bin/bash","/data/scripts/run.sh"] - name: DLC-WEBTEST--WEBTEST1 value: "true" ports: - containerPort: 80 readinessProbe: exec: command: - curl - http://webtest-static.test.com/index.html - -x - "127.0.0.1:80" initialDelaySeconds: 20 periodSeconds: 5 successThreshold: 1 --- apiVersion: v1 kind: Service metadata: name: webtest1-svc labels: app: webtest-static-test-com test: test1 spec: ports: - port: 80 targetPort: 80 protocol: TCP name: http selector: app: webtest-static-test-com