目錄
一、pod基本管理
Pod是Kubernetes創建或部署的最小/最簡單的基本單位，一個Pod代表集羣上正在運行的一個進程。
一個Pod封裝一個應用容器（也可以有多個容器），存儲資源、一個獨立的網絡IP以及管理控制容器運行方式的策略選項。
Pod代表部署的一個單位：Kubernetes中單個應用的實例，它可能由單個容器或多個容器共享組成的資源。
1、資源限制
#Kubernetes實現資源限制，對每個Pod設置資源限制
Pod.spec.containers.resources 資源限制
cat nginx-resource.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx-pod
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
resources:
requests:
memory: "64Mi"
cpu: "250m"
limits:
memory: "128Mi"
cpu: "500m"
2、調度約束
#Kubernetes調度約束，將Pod調度到某一固定的後端node節點上
Pod.spec.nodeName 強制約束Pod調度到指定Node節點上
Pod.spec.nodeSelector 通過lable-selector機制選擇節點
cat nginx-node-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-deploymeng-node
spec:
replicas: 3
template:
metadata:
name: nginx-pod
labels:
app: nginx
spec:
nodeName: 192.168.192.129 ##pod部署在192.168.192.129節點上
#nodeSelector: ##pod部署在標籤爲app: nginx的節點上
#app: nginx
containers:
- name: test
image: nginx:1.10
ports:
- containerPort: 80
3、重啓策略
Pod的重啓策略（RestartPolicy）應用與Pod內所有容器，並且僅在Pod所處的Node上由kubelet進行判斷和重啓操作。當某個容器異常退出或者健康檢查失敗時，kubelet將根據RestartPolicy的設置來進行相應的操作。。
支持三種策略：
Always：當容器終止退出後，總是重啓容器，默認策略。
OnFailure：當容器異常退出（退出狀態碼非0）時，才重啓容器。
Never：當容器終止退出，從不重啓容器
cat nginx-node.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-test
labels:
test: centos
spec:
containers:
- name: hello
image: centos:6
command: ["bash","-c","while true;do date;sleep 1;done"]
restartPolicy: OnFilure
4、健康檢查
#Kubernetes健康檢查，在Pod啓動後進行檢測
livenessProbe
如果檢查失敗，將殺死容器，根據Pod的restartPolicy來操作。
readinessProbe
如果檢查失敗，Kubernetes會把Pod從service endpoints中剔除。
Probe支持以下三種檢查方法：
httpGet
發送HTTP請求，返回200-400範圍狀態碼爲成功。
exec
執行Shell命令返回狀態碼是0爲成功。
tcpSocket
發起TCP Socket建立成功。
4.1、httpGet，訪問固定頁面，若能夠訪問則判斷改Pod爲健康，否則重啓，對Pod節點中的index.html文件進行健康監測。
啓動該yaml文件，注意，Service服務一直開啓着，要不無法訪問該Pod。
initialDelaySeconds: 10 表示在Pod啓動10秒後進行檢測。
periodSeconds: 5 表示進行健康監測的頻率爲5秒1次。
timeoutSeconds: 3 表示健康監測失敗後的超時時長。
cat nginx-health.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx-health
labels:
app: nginx
spec:
containers:
- name: nginx-health
image: nginx:1.10
livenessProbe:
#tcpSocket:
httpGet:
path: /index.html
port: 80
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
ports:
- containerPort: 80
可以試着將index.html刪除，或換一個文件路徑，這樣使用kubectl describe pod nginx-health就會顯示unhealthy
4.2、tcpSocket，通過檢測80，或其他端口是否正常來判斷Pod的健康狀態。
cat nginx-health.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx-health
labels:
app: nginx
spec:
containers:
- name: nginx-health
image: nginx:1.10
livenessProbe:
tcpSocket:
port: 80
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
ports:
- containerPort: 80
#volumes:
4.3、exec，通過命令行的形式進行健康狀態檢測。
[root@node-1 k8s-yaml]# cat nginx-health-exec.yaml
apiVersion: v1
kind: Pod
metadata:
labels:
test: liveness
name: liveness-exec
spec:
containers:
- name: liveness
image: nginx:1.10
livenessProbe:
exec:
command:
- cat
- /usr/share/nginx/html/index.html
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
二、server管理
server主要將用戶請求轉發到pod，對外提供負載均衡入口
1、服務創建
[root@manage01 yaml]# cat server-create.yaml
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
ports:
- name: http
protocol: TCP
port: 10080
targetPort: 80
- name: https
protocol: TCP
port: 10443
targetPort: 443
selector:
app: nginx
[root@manage01 yaml]# kubectl create -f server-create.yaml
查看service和endpoints
[root@manage01 yaml]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.10.10.1 <none> 443/TCP 8d
my-service ClusterIP 10.10.10.242 <none> 10080/TCP,10443/TCP 5m17s
nginx-service ClusterIP 10.10.10.134 <none> 18080/TCP 170m
[root@manage01 yaml]# kubectl get ep
NAME ENDPOINTS AGE
kubernetes 192.168.192.128:6443 8d
my-service 172.18.40.4:80,172.18.40.5:80,172.18.40.4:443 + 1 more... 5m21s
nginx-service 172.18.40.4:80,172.18.40.5:80 170m
2、服務發現
假設集羣有兩組組件，第一組組件負載均衡地址爲10.10.10.242，現在二組組件副本
希望訪問一組組件，那就需要訪問對用vip，服務發現就是用來獲取VIP地址。
基於DNS的服務發現是不受Service資源所在的名稱空間和創建時間的限制。
#我在創建kube-dns服務時先採用官方發佈最新版本1.14.13，啓動出現很多問題，解決後成功啓動，發現其中一個Pod遇到CrashLoopBackOff錯誤，暫時沒找到原因，爲了不影響日後，換做1.14.10版本正常啓動，先將1.14.13版本遇到問題FAQ和1.14.10版本配置文件記錄。
2.1、 kube-dns-1.14.13版本FAQ
NO.1：
[root@manage01 yaml]# https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/kube-dns/kube-dns.yaml.sed
clusterIP: 10.10.10.2
images: XXXXX.aliyun.XXXX
解決：
官方配置文件可能下載失敗，科學上網ok；clusterIP地址需要修改爲kubelet的配置文件中定義的cluster-dns地址（集羣DNS服務的工作地址）；三個images鏡像需要修改爲對應版本的阿里雲鏡像。
NO.2:
Error from server (BadRequest): error when creating "kube-dns.yaml": Deployment in version "v1" cannot be handled as a Deployment: v1.Deployment.Spec: v1.DeploymentSpec.Template: v1.PodTemplateSpec.Spec: v1.PodSpec.Containers: []v1.Container: v1.Container.Resources: v1.ResourceRequirements.Requests: Limits: unmarshalerDecoder: quantities must match the regular expression '^([+-]?[0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$', error found in #10 byte of ...|ORY_LIMIT"},"request|..., bigger context ...|resources":{"limits":{"memory":"$DNS_MEMORY_LIMIT"},"requests":{"cpu":"100m","memory":"70Mi"}},"secu|...
error parsing kube-dns.yaml: error converting YAML to JSON: yaml: line 57: did not find expected key
解決：
limits:
memory: $DNS_MEMORY_LIMIT
[root@manage01 yaml]# echo $DNS_MEMORY_LIMIT
跟57行沒什麼關係，原因在於配置文件設置資源限制，memory值採用變量，但是我echo並沒有內容，改爲memory: 200Mi具體值即可。
NO.3
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsUser: 1001
runAsGroup: 1001
此時pod.Spec.SecurityContext.SupplementalGroups is forbidden
&&
Warning FailedCreate 3s (x5 over 41s) replicaset-controller (combined from similar events): Error creating: pods "kube-dns-696f8cf5dc-9m2bc" is forbidden: SecurityContext.RunAsUser is forbidden
解決：
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsUser: 1001
runAsGroup: 1001
這兩個錯誤報 runAsUser: 1001，runAsGroup: 1001，個人不理解1001什麼意思，暫時註釋掉。
2.2 、kube-dns：1.14.10配置文件
[root@manage01 yaml]# cat kube-dns.yaml
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Should keep target in cluster/addons/dns-horizontal-autoscaler/dns-horizontal-autoscaler.yaml
# in sync with this file.
# Warning: This is a file generated from the base underscore template file: kube-dns.yaml.base
apiVersion: v1
kind: Service
metadata:
name: kube-dns
namespace: kube-system
labels:
k8s-app: kube-dns
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
kubernetes.io/name: "KubeDNS"
spec:
selector:
k8s-app: kube-dns
clusterIP: 10.10.10.2
ports:
- name: dns
port: 53
protocol: UDP
- name: dns-tcp
port: 53
protocol: TCP
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: kube-dns
namespace: kube-system
labels:
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
---
apiVersion: v1
kind: ConfigMap
metadata:
name: kube-dns
namespace: kube-system
labels:
addonmanager.kubernetes.io/mode: EnsureExists
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: kube-dns
namespace: kube-system
labels:
k8s-app: kube-dns
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
spec:
# replicas: not specified here:
# 1. In order to make Addon Manager do not reconcile this replicas parameter.
# 2. Default is 1.
# 3. Will be tuned in real time if DNS horizontal auto-scaling is turned on.
strategy:
rollingUpdate:
maxSurge: 10%
maxUnavailable: 0
selector:
matchLabels:
k8s-app: kube-dns
template:
metadata:
labels:
k8s-app: kube-dns
annotations:
scheduler.alpha.kubernetes.io/critical-pod: ''
spec:
tolerations:
- key: "CriticalAddonsOnly"
operator: "Exists"
volumes:
- name: kube-dns-config
configMap:
name: kube-dns
optional: true
containers:
- name: kubedns
image: registry.cn-hangzhou.aliyuncs.com/google_containers/k8s-dns-kube-dns-amd64:1.14.10
resources:
# TODO: Set memory limits when we've profiled the container for large
# clusters, then set request = limit to keep this container in
# guaranteed class. Currently, this container falls into the
# "burstable" category so the kubelet doesn't backoff from restarting it.
limits:
memory: 170Mi
requests:
cpu: 100m
memory: 70Mi
livenessProbe:
httpGet:
path: /healthcheck/kubedns
port: 10054
scheme: HTTP
initialDelaySeconds: 60
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 5
readinessProbe:
httpGet:
path: /readiness
port: 8081
scheme: HTTP
# we poll on pod startup for the Kubernetes master service and
# only setup the /readiness HTTP server once that's available.
initialDelaySeconds: 3
timeoutSeconds: 5
args:
- --domain=cluster.local.
- --dns-port=10053
- --config-dir=/kube-dns-config
- --v=2
env:
- name: PROMETHEUS_PORT
value: "10055"
ports:
- containerPort: 10053
name: dns-local
protocol: UDP
- containerPort: 10053
name: dns-tcp-local
protocol: TCP
- containerPort: 10055
name: metrics
protocol: TCP
volumeMounts:
- name: kube-dns-config
mountPath: /kube-dns-config
- name: dnsmasq
image: registry.cn-hangzhou.aliyuncs.com/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.10
livenessProbe:
httpGet:
path: /healthcheck/dnsmasq
port: 10054
scheme: HTTP
initialDelaySeconds: 60
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 5
args:
- -v=2
- -logtostderr
- -configDir=/etc/k8s/dns/dnsmasq-nanny
- -restartDnsmasq=true
- --
- -k
- --cache-size=1000
- --no-negcache
- --dns-loop-detect
- --log-facility=-
- --server=/cluster.local/127.0.0.1#10053
- --server=/in-addr.arpa/127.0.0.1#10053
- --server=/ip6.arpa/127.0.0.1#10053
ports:
- containerPort: 53
name: dns
protocol: UDP
- containerPort: 53
name: dns-tcp
protocol: TCP
# see: https://github.com/kubernetes/kubernetes/issues/29055 for details
resources:
requests:
cpu: 150m
memory: 20Mi
volumeMounts:
- name: kube-dns-config
mountPath: /etc/k8s/dns/dnsmasq-nanny
- name: sidecar
image: registry.cn-hangzhou.aliyuncs.com/google_containers/k8s-dns-sidecar-amd64:1.14.10
livenessProbe:
httpGet:
path: /metrics
port: 10054
scheme: HTTP
initialDelaySeconds: 60
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 5
args:
- --v=2
- --logtostderr
- --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local,5,SRV
- --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local,5,SRV
ports:
- containerPort: 10054
name: metrics
protocol: TCP
resources:
requests:
memory: 20Mi
cpu: 10m
dnsPolicy: Default # Don't use cluster DNS.
serviceAccountName: kube-dns
2.3、 kube-dns啓動、查看、測試
#期望不報錯吧
[root@manage01 yaml]# kubectl create -f kube-dns.yaml
service/kube-dns created
serviceaccount/kube-dns created
configmap/kube-dns created
deployment.apps/kube-dns created
[root@manage01 yaml]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
kube-dns-c56c79458-5pvxk 3/3 Running 0 70m
#找個容器解析域名試試
[root@manage01 yaml]# kubectl run busybox --image busybox:1.28 --restart=Never -it busybox -- nslookup kubernetes.default
Server: 10.10.10.2
Address 1: 10.10.10.2 kube-dns.kube-system.svc.cluster.local
Name: kubernetes.default
Address 1: 10.10.10.1 kubernetes.default.svc.cluster.local
#系統初始化時會默認將cluster.local和主機所在域作爲DNS的本地域使用，這些文件會在Pod創建時以DNS配置的相關信息注入它的/etc/resolv.conf文件中。
root@nginx-deployment-97dd56676-7pkrr:/# cat /etc/resolv.conf
nameserver 10.10.10.2
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5