K8S的重要性不多說了。今天開始學習K8S，安裝階段竟然遇到很多問題，終於解決了，將安裝方法總結下：
1.安裝centos 7，如果最小化安裝centos，執行
(1)容器內初始安裝：yum install -y procps iproute2 net-tools vim
(2)取消firewalld :
systemctl disable firewalld
systemctl stop firewalld
(3)禁用SELinux 修改/etc/sysconfig/selinux，將SELINUX=enforcing修改爲SELINUX=disabled，然後setenforce 0
2.centos安裝docker：
安裝依賴包：yum install -y yum-utils device-mapper-persistent-data lvm2
# 添加Docker軟件包源
yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
# 安裝Docker CE
yum install docker-ce -y
# 啓動
systemctl start docker
# 開機啓動
systemctl enable docker
# 查看Docker信息
docker info
配置dockerhub加速器
curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://bc437cce.m.daocloud.io
3. 修改本地鏡像源：/etc/yum.repos.d/k8s.repo
[kubernetes]
name=Kubernetes Repository
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
然後運行yum install 命令安裝kubeadm和相關工具:
#yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
systemctl start kubelet
systemctl enable kubelet
4.在 執行kubeadm init 的時候會下載類似k8s.gcr.io/kube-apiserver:v1.17.0的鏡像，因爲國內封的原因，到google的k8s.gcr.io不可達報錯。所以提前從docker.io下載，並修改tag。
|
錯誤信息：[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-apiserver:v1.17.0: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-controller-manager:v1.17.0: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-scheduler:v1.17.0: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-proxy:v1.17.0: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/etcd:3.4.3-0: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/coredns:1.6.5: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) , error: exit status 1 |
提前下載鏡像：
docker pull mirrorgooglecontainers/kube-apiserver:v1.17.0-alpha.0
docker pull mirrorgooglecontainers/kube-controller-manager:v1.17.0-alpha.0
docker pull mirrorgooglecontainers/kube-scheduler:v1.16.0-beta.1
docker pull mirrorgooglecontainers/kube-proxy:v1.17.0-alpha.0
docker pull mirrorgooglecontainers/pause:latest
docker pull mirrorgooglecontainers/etcd:3.4.2-0
docker pull coredns/coredns:latest
將鏡像tag到kubelet init中要求的鏡像名稱：
docker tag docker.io/mirrorgooglecontainers/kube-apiserver:v1.17.0-alpha.0 k8s.gcr.io/kube-apiserver:v1.17.0
docker tag docker.io/mirrorgooglecontainers/kube-controller-manager:v1.17.0-alpha.0 k8s.gcr.io/kube-controller-manager:v1.17.0
docker tag docker.io/mirrorgooglecontainers/kube-scheduler:v1.16.0-beta.1 k8s.gcr.io/kube-scheduler:v1.17.0
docker tag docker.io/mirrorgooglecontainers/kube-proxy:v1.17.0-alpha.0 k8s.gcr.io/kube-proxy:v1.17.0
docker tag docker.io/mirrorgooglecontainers/pause:latest k8s.gcr.io/pause:3.1
docker tag docker.io/mirrorgooglecontainers/etcd:3.4.2-0 k8s.gcr.io/etcd:3.4.3-0
docker tag docker.io/coredns/coredns:latest k8s.gcr.io/coredns:1.6.5
5.執行kubeadm init，繼續報錯，多個錯誤，一個一個看：
|
W1230 19:06:11.581755 103362 validation.go:28] Cannot validate kube-proxy config - no validator is available W1230 19:06:11.583182 103362 validation.go:28] Cannot validate kubelet config - no validator is available [init] Using Kubernetes version: v1.17.0 [preflight] Running pre-flight checks [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/ error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1 [ERROR Swap]: running with swap on is not supported. Please disable swap [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...` To see the stack trace of this error execute with --v=5 or higher |
(1)[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
error execution phase preflight: [preflight] Some fatal errors occurred:
通過在/etc/docker/daemon.json中增加：
|
{ "exec-opts": ["native.cgroupdriver=systemd"] } |
並重啓docker：systemctl restart docker
(2)[ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1
通過執行如下命令解決：
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
長期的：
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system
(3)[ERROR Swap]: running with swap on is not supported. Please disable swap
通過執行如下命令解決：
swapoff -a
長期：sed -i "s/\/dev\/mapper\/centos-swap/\#\/dev\/mapper\/centos-swap/g" /etc/fstab
6.此時重置一下 kubeadm 之前執行過的操作，再進行初始化
kubeadm reset && systemctl start kubelet && kubeadm init --ignore-preflight-errors=all
出現如下信息，成功：
|
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.239.128:6443 --token j6aqes.tey9pet7vux0j9q2 \ --discovery-token-ca-cert-hash sha256:7fbc59430a88edd2c7455f61bc9282d691c5c914b80875683fc547163c51e398 |
7.依據提示執行（複製命令到普通用戶的home目錄下）：
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
這樣就在Master上安裝了kubenets，但在集羣內還是沒有可用的工作Node，並缺乏容器網絡的配置
可以執行kubectl get -n kube-system configmap
後記：後續又做了一次初始化，卻總是出現如下錯誤，
|
Unfortunately, an error has occurred: This error is likely caused by: If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands: Additionally, a control plane component may have crashed or exited when started by the container runtime. |
通過設置了代理解決
export HTTP_PROXY=http://hgc1.host.myzone.fun:574
export HTTPS_PROXY=$HTTP_PROXY
export NO_PROXY="192.168.239.128" //不代理本機，很重要
docker 並不會使用系統的環境變量，需要對 docker 單獨進行配置。
mkdir -p /etc/systemd/system/docker.service.d
touch /etc/systemd/system/docker.service.d/http-proxy.conf
echo -e '[Service]\nEnvironment="HTTP_PROXY=http://hgc1.host.myzone.fun:574"' > /etc/systemd/system/docker.service.d/http-proxy.conf
touch /etc/systemd/system/docker.service.d/https-proxy.conf
echo -e '[Service]\nEnvironment="HTTPS_PROXY=https://hgc1.host.myzone.fun:574"' > /etc/systemd/system/docker.service.d/http-proxy.conf
重啓 docker 使配置生效
systemctl daemon-reload && systemctl restart docker
docker info | grep -i proxy
HTTP Proxy: http://hgc1.host.myzone.fun:574
HTTPS Proxy: https://hgc1.host.myzone.fun:574
完成可以看到docker啓動的進程，
備註：該文章有參考https://blog.csdn.net/u012570862/article/details/80150988
8.節點kubeadm join 遇到很多問題，如下錯誤，查看kubelet無法啓動
| [root@localhost docker]# kubeadm join 192.168.239.128:6443 --token jv5na2.e3atk25qcqs372ux --discovery-token-ca-cert-hash sha256:d2d21911885b6682e5557d264f88fa926d917730f44973933cc2ee56d1e26597 --ignore-preflight-errors=all W0101 02:36:18.667712 79434 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set. [preflight] Running pre-flight checks [WARNING FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists [WARNING FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1 [WARNING FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1 [WARNING Port-10250]: Port 10250 is in use [WARNING FileAvailable--etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt already exists [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system namespace [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... [kubelet-check] Initial timeout of 40s passed. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused. error execution phase kubelet-start: error uploading crisocket: timed out waiting for the condition To see the stack trace of this error execute with --v=5 or higher |
先執行kubeadm reset，然後執行kubeadm join 192.168.239.128:6443 --token jv5na2.e3atk25qcqs372ux --discovery-token-ca-cert-hash sha256:d2d21911885b6682e5557d264f88fa926d917730f44973933cc2ee56d1e26597 --ignore-preflight-errors=all
節點加入集羣成功
接下來發現kubectl get nodes出現錯誤，需要從master上將admin.conf拷貝到nodes上，並做成環境變量KUBECONFIG
[root@localhost kubernetes]# kubectl get nodes
W0101 05:15:03.349748 28952 loader.go:223] Config not found: /etc/kubernetes/admin.conf
The connection to the server localhost:8080 was refused - did you specify the right host or port?
scp [email protected]:/etc/kubernetes/admin.conf /etc/kubernetes/.
echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
source ~/.bash_profile