1.兩種探針
readiness probe(就緒探針)
監測容器是否就緒?只有pod裏的容器就緒,kubelet纔會認爲pod處於就緒狀態.
就緒探針的作用是控制哪些pod可以作爲svc的後端,如果pod不是就緒狀態,就把它從svc load balancer中移除.
liveness probe(存活探針)
監測容器是否存活?如果容器中的應用出現問題,liveness將檢測到容器不健康會通知kubelet,kubelet重啓該pod容器.
2.使用探針的三種方式
官網介紹了三種,見下:
command命令執行
http request訪問
tcp socket連接
個人比較喜歡用第三種方式,tcp socket.
3.tcp socket方式學習測試
tcp socket方式
這個方式比較好理解.
比如說,起一個nginx容器,nginx服務提供的端口是80端口.
配置tcp socket 探針,設定隔一個時間,用tcp方式連接80端口,如果連接成功,就返回容器健康或者就緒,如果連接失敗,返回容器不健康或者不就緒,kubelet重啓容器.
逆向思維示例:
簡單思路:探針tcp socket連接不存在的8080端口,必然連接失敗報錯,從而實現pod不斷重啓.
[root@k8s-master1 k8s_tanzhen]# cat tcp_ness
apiVersion: v1
kind: Pod
metadata:
name: httpd
labels:
app: httpd
spec:
containers:
- name: httpd
image: nginx
ports:
- containerPort: 80
readinessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 45
periodSeconds: 20
livenessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 45
periodSeconds: 20
[root@k8s-master1 k8s_tanzhen]#起一個nginx的pod容器,提供服務端口80.
配置探針連接端口8080,第一次監測時間爲pod容器啓動後的45s,第一次監測後每隔20s監測一次.
測試結果,pod容器一直在重啓.
[root@k8s-master1 k8s_tanzhen]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE
httpd 0/1 CrashLoopBackOff 7 18m 172.30.35.3 k8s-master3describe報錯
Warning Unhealthy 6m (x19 over 16m) kubelet, k8s-master3 Liveness probe failed: dial tcp 172.30.35.3:8080: connect: connection refused
Warning Unhealthy 2m (x15 over 16m) kubelet, k8s-master3 Readiness probe failed: dial tcp 172.30.35.3:8080: connect: connection refused探針自動tcp連接容器ip:8080端口,失敗.所以容器一直重啓.
正常配置示例
正常配置是連接提供服務的80端口
簡單思路:理論上來說,長時間運行的應用程序最終會過渡到中斷狀態,除非重新啓動,否則無法恢復.Kubernetes提供了活性探針來檢測和補救這種情況.這是配置探針的根本原因,以防萬一.
[root@k8s-master1 k8s_tanzhen]# cat tcp_ness
apiVersion: v1
kind: Pod
metadata:
name: httpd
labels:
app: httpd
spec:
containers:
- name: httpd
image: nginx
ports:
- containerPort: 80
readinessProbe:
tcpSocket:
port: 80
initialDelaySeconds: 45
periodSeconds: 20
livenessProbe:
tcpSocket:
port: 80
initialDelaySeconds: 45
periodSeconds: 20[root@k8s-master1 k8s_tanzhen]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE
httpd 1/1 Running 0 2m 172.30.35.3 k8s-master3正常配置模擬測試案例
簡單思路:起nginx容器,然後執行命令殺死nginx進程,設定探針監測連接tcp socket 80端口,當nginx進程被殺死後,tcp socket連接失敗,探針監測容器爲不健康不就緒,kubelet重啓容器.
[root@k8s-master1 k8s_tanzhen]# cat tcp_ness
apiVersion: v1
kind: Pod
metadata:
name: httpd
labels:
app: httpd
spec:
containers:
- name: httpd
image: nginx
args:
- /bin/sh
- -c
- sleep 60;nginx -s quit
ports:
- containerPort: 80
readinessProbe:
tcpSocket:
port: 80
initialDelaySeconds: 20
periodSeconds: 10
livenessProbe:
tcpSocket:
port: 80
initialDelaySeconds: 20
periodSeconds: 10
[root@k8s-master1 k8s_tanzhen]#配置參數說明:
容器啓動後,執行nginx -s quit殺死Nginx進程
容器啓動20s後開始執行readiness和liveness檢測
容器啓動後35s左右
探針監測到nginx進程已經死掉,無法連接到80端口,報警見下:
Warning Unhealthy 8s (x3 over 28s) kubelet, k8s-master3 Liveness probe failed: dial tcp 172.30.35.3:80: connect: connection refused
Warning Unhealthy 7s (x3 over 27s) kubelet, k8s-master3 Readiness probe failed: dial tcp 172.30.35.3:80: connect: connection refused整個重啓事件記錄
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m default-scheduler Successfully assigned default/httpd to k8s-master3
Normal Pulled 1m (x2 over 2m) kubelet, k8s-master3 Successfully pulled image "nginx"
Normal Created 1m (x2 over 2m) kubelet, k8s-master3 Created container
Normal Started 1m (x2 over 2m) kubelet, k8s-master3 Started container
Warning Unhealthy 16s (x6 over 1m) kubelet, k8s-master3 Liveness probe failed: dial tcp 172.30.35.3:80: connect: connection refed
Warning Unhealthy 15s (x8 over 1m) kubelet, k8s-master3 Readiness probe failed: dial tcp 172.30.35.3:80: connect: connection resed
Normal Pulling 5s (x3 over 2m) kubelet, k8s-master3 pulling image "nginx"
Normal Killing 5s (x2 over 1m) kubelet, k8s-master3 Killing container with id docker://httpd:Container failed liveness prob. Container will be killed and recreated.可以看到,nginx進程殺死後,pod自動重啓.
[root@k8s-master1 k8s_tanzhen]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE
httpd 0/1 Running 4 5m 172.30.35.3 k8s-master3實現測試目的