kubeadm方式安裝kuberneters1.17.1(centos7.5)

• 首先準備兩臺虛擬機

虛擬機配置如下：

kube-master    192.168.9.199      CentOS Linux release 7.5.1804    docker-ce-18.09.7、kuberneters1.17.1
kube-node1     192.168.9.198       CentOS Linux release 7.5.1804    docker-ce-18.09.7、kuberneters1.17.1

• 以下在每個主機都要執行

1、修改主機名，在 /etc/hosts 中添加三個結點的主機名解析

在master機子kube-master 執行hostnamectl set-hostname kube-master

在node機子kube-node1 執行hostnamectl set-hostname kube-node1

每臺機子都執行

echo '192.168.9.199 kube-master' >> /etc/hosts
echo '192.168.9.198 kube-node1' >> /etc/hosts

2、關閉防火牆、設置SeLinux、關閉swap

[root@kube-master ~]# systemctl stop firewalld
[root@kube-master ~]# systemctl disable firewalld
[root@kube-master ~]# setenforce 0
[root@kube-master ~]# sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
[root@kube-master ~]# swapoff -a
[root@kube-master ~]# cp /etc/fstab /etc/fstab_tmp
[root@kube-master ~]# cat /etc/fstab_tmp |grep -v swap > /etc/fstab
[root@kube-master ~]# yum install -y wget

3、安裝docker-ce-18.09.7
獲取阿里雲的鏡像加速器步驟：

(1)註冊阿里雲賬號，註冊地址 https://promotion.aliyun.com/ntms/act/kubernetes.html ，註冊成功後登陸。
(2)進入https://www.aliyun.com/?spm=5176.202918.fnqwg5agi.2.7bab4ef5LT14z5，依次點擊【產品分類】----->【雲計算基礎】------>【容器鏡像服務】

(3)如下圖，點擊【管理控制檯】

（4）進入阿里雲容器鏡像服務頁面，點擊【鏡像加速器】，複製複製加速器地址。

獲取後到的加速器替換下列的XXXXX

 

安裝docker步驟：

#卸載原有docker

[root@kube-master ~]# yum remove -y docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-selinux docker-engine-selinux docker-engine

#開始安裝

[root@kube-master ~]# yum install -y yum-utils device-mapper-persistent-data lvm2

#docker倉庫

[root@kube-master ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root@kube-master ~]# yum install -y docker-ce-18.09.7 docker-ce-cli-18.09.7 containerd.io

#設置阿里雲的docker鏡像加速器 地址獲取參考

[root@kube-master ~]# cat <<EOF >/etc/docker/daemon.json
{
  "registry-mirrors": ["https://XXXXX.mirror.aliyuncs.com"] 
}
EOF

#啓動docker服務

[root@kube-master ~]# systemctl start docker

#驗證

[root@kube-master ~]# docker version

Client:
 Version:           18.09.7
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        2d0083d
 Built:             Thu Jun 27 17:56:06 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.7
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       2d0083d
  Built:            Thu Jun 27 17:26:28 2019
  OS/Arch:          linux/amd64
  Experimental:     false

#設置開機啓動

[root@kube-master ~]# systemctl enable docker

備註：如遇啓動不起來，可通過 systemctl status docker 或者 journalctl -xe -u docker 命令查看具體問題。

 

4、安裝kubernetes
配置/etc/sysctl.conf
vi /etc/sysctl.conf 末行添加

net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

保存後執行sysctl -p 生效

#刪除之前裝過的kubernetes

[root@kube-master ~]# yum remove -y kubelet kubeadm kubectl

#添加k8s的源

[root@kube-master ~]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
       http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

#安裝

[root@kube-master ~]# yum install -y kubelet-1.17.1 kubeadm-1.17.1 kubectl-1.17.1

#修改docker的Cgroup Driver 爲systemd

[root@kube-master ~]# sed -i "s#^ExecStart=/usr/bin/dockerd.*#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --exec-opt native.cgroupdriver=systemd#g" /usr/lib/systemd/system/docker.service

 

[root@kube-master ~]# systemctl daemon-reload
[root@kube-master ~]# systemctl restart docker
[root@kube-master ~]# systemctl enable kubelet

 

• 初始化 master （kube-master），僅在master節點執行

[root@kube-master ~]# export MASTER_IP=192.168.9.199  #master的宿主ip地址
[root@kube-master ~]# export APISERVER_NAME=kube-master #master的hostname
[root@kube-master ~]# export POD_SUBNET=10.20.0.1/16  #pod 的subnet
[root@kube-master ~]# echo "${MASTER_IP}    ${APISERVER_NAME}" >> /etc/hosts

kubeadm 初始化配置文件創建

[root@kube-master ~]# rm -f ./kubeadm-config.yaml

 

[root@kube-master ~]# cat <<EOF > ./kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.17.1
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
controlPlaneEndpoint: "${APISERVER_NAME}:6443"
networking:
  serviceSubnet: "10.10.0.0/16"
  podSubnet: "${POD_SUBNET}"
  dnsDomain: "cluster.local"
EOF

#準備鏡像
因國內訪問不到goole服務器，只能拉取國內的鏡像，這裏以阿里云爲例。

準備腳本pullK8s.sh

#!/bin/bash
KUBE_VERSION=v1.17.4
KUBE_PAUSE_VERSION=3.1
ETCD_VERSION=3.4.3-0
DNS_VERSION=1.6.5
username=registry.cn-hangzhou.aliyuncs.com/google_containers
images=(kube-proxy-amd64:${KUBE_VERSION}
kube-scheduler-amd64:${KUBE_VERSION}
kube-controller-manager-amd64:${KUBE_VERSION}
kube-apiserver-amd64:${KUBE_VERSION}
pause:${KUBE_PAUSE_VERSION}
etcd-amd64:${ETCD_VERSION}
coredns:${DNS_VERSION}
    )
for image in ${images[@]}
do
    docker pull ${username}/${image}
    docker tag ${username}/${image} k8s.gcr.io/${image}
    #docker tag ${username}/${image} gcr.io/google_containers/${image}
    docker rmi ${username}/${image}
done

執行腳本

[root@kube-master ~]# sh pullK8s.sh

查看鏡像

[root@kube-master ~]# docker images

#執行init

[root@kube-master ~]# kubeadm init --config=kubeadm-config.yaml --upload-certs

一切正常後會顯示類似如下信息

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join kube-master:6443 --token csakso.fjzahtnog7h05i7x \
    --discovery-token-ca-cert-hash sha256:64c3e4924b1a1fe866259c7cfa4036cab533a5c5e68b74d59cb341dcd8e774df \
    --control-plane --certificate-key ebcf32bbc84a183f50c4aaf32b3d370288b0806da0e7c70f1af8a6b4f6c38823

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join kube-master:6443 --token 8z507c.c2e35zt1fa43jah6 \
    --discovery-token-ca-cert-hash sha256:f91bc51f1b7806a4d9b9bc8d0063c1a67008e5ed92f46773d2187f7b7dd66697

如果忘記token 後續也可以在master節點上執行kubeadm token create --print-join-command 重新獲取join token

安裝calico3.10.2

[root@kube-master ~]# wget https://docs.projectcalico.org/v3.10/manifests/calico.yaml
[root@kube-master ~]# sed -i "s#192\.168\.0\.0/16#${POD_SUBNET}#" calico.yaml
[root@kube-master ~]# kubectl apply -f calico.yaml 

#發現報錯
The connection to the server kube-master:6443 was refused - did you specify the right host or port?

排查錯誤

[root@kube-master ~]# systemctl status kubelet
[root@kube-master ~]# journalctl -xeu kubelet

#有個錯誤
 Failed to start ContainerManager failed to initialize top level QOS containers: failed to update top level Burstable QOS cgroup : failed to set supported cgroup subsystems for cgroup [kubepods burstable]: failed to find subsystem mount for required subsystem:pids


因此修改kubelet的啓動配置文件 /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf ，在ExecStart上添加 --feature-gates SupportPodPidsLimit=false --feature-gates SupportNodePidsLimit=false，修改後執行systemctl daemon-reload && systemctl restart kubelet。至此，kubelet已經能成功啓動。

如果以上方法試過還不行，可以試着以下方式：

[root@kube-master ~]# mkdir -p $HOME/.kube
[root@kube-master ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@kube-master ~]# chown $(id -u):$(id -g) $HOME/.kube/config

 

#下載需要一段時間 稍等一會 等ready 都爲1時就行

[root@kube-master ~]# kubectl apply -f calico.yaml 

 

 

• worker 節點 (kube-node1)，僅在node節點執行

[root@kube-node1 ~]# kubeadm join kube-master:6443 --token 8z507c.c2e35zt1fa43jah6 --discovery-token-ca-cert-hash sha256:f91bc51f1b7806a4d9b9bc8d0063c1a67008e5ed92f46773d2187f7b7dd66697

驗證
 

[root@kube-master ~]# kubectl get nodes
NAME         STATUS   ROLES    AGE     VERSION
kube-master   Ready    master   7d19h   v1.17.1
kube-node1    Ready    <none>   6d23h   v1.17.1

[root@kube-master ~]# kubectl get pod -A
NAMESPACE     NAME                                         READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-7489ff5b7c-6tjmc     1/1     Running   4          7d19h
kube-system   calico-node-s2mng                            1/1     Running   4          7d19h
kube-system   calico-node-xk2vf                            1/1     Running   0          6d23h
kube-system   coredns-7f9c544f75-f2cnz                     1/1     Running   11         7d19h
kube-system   coredns-7f9c544f75-m85zj                     1/1     Running   9          7d19h
kube-system   dashboard-metrics-scraper-65f454dff8-kgvk7   1/1     Running   1          6d19h
kube-system   etcd-kube-master                              1/1     Running   5          7d19h
kube-system   kube-apiserver-kube-master                    1/1     Running   126        7d19h
kube-system   kube-controller-manager-kube-master           1/1     Running   316        7d19h
kube-system   kube-proxy-mxhg5                             1/1     Running   0          6d23h
kube-system   kube-proxy-t85h8                             1/1     Running   4          7d19h
kube-system   kube-scheduler-kube-master                    1/1     Running   338        7d19h
kube-system   kubernetes-dashboard-7bf47cd79c-g62jn        1/1     Running   21         6d19h

 問題列舉:

• 1、證書過期問題

kubeadm join證書過期
過段時間執行kubeadm join再將新節點加入集羣的時候會報下面的錯誤
unable to fetch the kubeadm-config ConfigMap: failed to get config map: Unauthorized

遇到這個問題我們需要新建token

[root@kube-master ~]# kubeadm token create

創建好後，查看剛纔創建的token

[root@kube-master ~]# kubeadm token list

使用列表中的token，替換上面命令的–token

[root@kube-node1 ~]# kubeadm join kube-master:6443 --token 8z507c.c2e35zt1fa43jah6 --discovery-token-ca-cert-hash sha256:f91bc51f1b7806a4d9b9bc8d0063c1a67008e5ed92f46773d2187f7b7dd66697

• 2、work節點如何執行kubectl命令

如果遇到錯誤提示：

The connection to the server localhost:8080 was refused - did you specify the right host or port?

將master節點的/etc/kubernetes/admin.conf拷貝到work節點相同目錄下
work節點添加環境變量

[root@kube-node1 ~]# export KUBECONFIG=/etc/kubernetes/admin.conf