聲明:
如果您有更好的技術與作者分享,或者商業合作;
請訪問作者個人網站 http://www.esqabc.com/view/message.html 留言給作者。
如果該案例觸犯您的專利,請在這裏:http://www.esqabc.com/view/message.html 留言給作者說明原由
作者一經查實,馬上刪除。
.
.
1、搭建前說明:
.
a、前提提條件、服務器,請查看這個地址:https://blog.csdn.net/esqabc/article/details/102726771
.
.
2、搭建kube-apiserver 高可用
- 使用Nginx 4層實現k8s節點(master節點和worker節點)高可用,訪問kube-apiserver的步驟
- 注意:搭建服務器,沒有特殊說明,一般默認在:k8s-01 操作
a、下載編譯nginx
(1)把下載好的文件上傳到:/opt/k8s/work
[root@k8s-01 ~]# cd /opt/k8s/work
解壓
[root@k8s-01 work]# tar -xzvf nginx-1.15.3.tar.gz
編譯
[root@k8s-01 work]# cd /opt/k8s/work/nginx-1.15.3
[root@k8s-01 nginx-1.15.3]# mkdir nginx-prefix
.
注意,下面命令有一個點的
.
[root@k8s-01 nginx-1.15.3]# ./configure --with-stream --without-http --prefix=$(pwd)/nginx-prefix --without-http_uwsgi_module
[root@k8s-01 nginx-1.15.3]# make && make install
說明一下:
- without-http_scgi_module --without-http_fastcgi_module
- with-stream:開啓 4 層透明轉發(TCP Proxy)功能;
- without-xxx:關閉所有其他功能,這樣生成的動態鏈接二進制程序依賴最小;
(2)創建目錄結構
[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 work]# source /opt/k8s/bin/environment.sh
for node_ip in ${NODE_IPS[@]}
do
echo ">>> ${node_ip}"
mkdir -p /opt/k8s/kube-nginx/{conf,logs,sbin}
done
(3)分發到其他主機
[root@k8s-01 ~]# cd /opt/k8s/work
for node_ip in ${NODE_IPS[@]}
do
echo ">>> ${node_ip}"
scp /opt/k8s/work/nginx-1.15.3/nginx-prefix/sbin/nginx root@${node_ip}:/opt/k8s/kube-nginx/sbin/kube-nginx
ssh root@${node_ip} "chmod a+x /opt/k8s/kube-nginx/sbin/*"
ssh root@${node_ip} "mkdir -p /opt/k8s/kube-nginx/{conf,logs,sbin}"
sleep 3
done
注意:如果出現下面錯誤,執行上面命令重複執行一次就OK:
出現下圖就代表成功:
(4)配置Nginx文件
[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 work]# cat > kube-nginx.conf <<EOF
添加下面內容:
worker_processes 1;
events {
worker_connections 1024;
}
stream {
upstream backend {
hash $remote_addr consistent;
server 172.26.16.249:6443 max_fails=3 fail_timeout=30s;
server 172.26.16.250:6443 max_fails=3 fail_timeout=30s;
server 172.26.16.251:6443 max_fails=3 fail_timeout=30s;
}
server {
listen *:8443;
proxy_connect_timeout 1s;
proxy_pass backend;
}
}
EOF
注意:只需要修改server 內容即可
.
(5)分發配置文件
[root@k8s-01 ~]# cd /opt/k8s/work
for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
scp kube-nginx.conf root@${node_ip}:/opt/k8s/kube-nginx/conf/kube-nginx.conf
done
(6)配置Nginx啓動文件
[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 work]# cat > kube-nginx.service <<EOF
添加下面內容:
[Unit]
Description=kube-apiserver nginx proxy
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=forking
ExecStartPre=/opt/k8s/kube-nginx/sbin/kube-nginx -c /opt/k8s/kube-nginx/conf/kube-nginx.conf -p /opt/k8s/kube-nginx -t
ExecStart=/opt/k8s/kube-nginx/sbin/kube-nginx -c /opt/k8s/kube-nginx/conf/kube-nginx.conf -p /opt/k8s/kube-nginx
ExecReload=/opt/k8s/kube-nginx/sbin/kube-nginx -c /opt/k8s/kube-nginx/conf/kube-nginx.conf -p /opt/k8s/kube-nginx -s reload
PrivateTmp=true
Restart=always
RestartSec=5
StartLimitInterval=0
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
(7)分發nginx啓動文件
[root@k8s-01 ~]# cd /opt/k8s/work
for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
scp kube-nginx.service root@${node_ip}:/etc/systemd/system/
done
(8)啓動 kube-nginx 服務
[root@k8s-01 ~]# cd /opt/k8s/work
for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
ssh root@${node_ip} "systemctl daemon-reload && systemctl enable kube-nginx && systemctl start kube-nginx"
done
(9)檢查 kube-nginx 服務運行狀態
[root@k8s-01 ~]# cd /opt/k8s/work
for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
ssh root@${node_ip} "systemctl status kube-nginx |grep 'Active:'"
done
出現下圖,就說明正常
3、KeepLived 部署
在所有master節點安裝keeplived
a、安裝keepalived
[root@k8s-01 ~]# yum install -y keepalived
b、配置keeplive服務
[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 work]# cat > /etc/keepalived/keepalived.conf <<EOF
添加下面內容
! Configuration File for keepalived
global_defs {
router_id 172.26.16.249
}
vrrp_script chk_nginx {
script "/etc/keepalived/check_port.sh 8443"
interval 2
weight -20
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 251
priority 100
advert_int 1
mcast_src_ip 172.26.16.249
nopreempt
authentication {
auth_type PASS
auth_pass 11111111
}
track_script {
chk_nginx
}
virtual_ipaddress {
172.26.16.252
}
}
EOF
注意:## 172.26.16.249 爲當前節點,172.26.16.253爲node節點
.
c、將配置拷貝到其他節點,並替換相關IP
[root@k8s-01 ~]# cd /opt/k8s/work
for node_ip in 172.26.16.249 172.26.16.250 172.26.16.251
do
echo ">>> ${node_ip}"
scp /etc/keepalived/keepalived.conf $node_ip:/etc/keepalived/keepalived.conf
done
d、替換IP
ssh root@172.26.16.250 sed -i 's#172.26.16.249#172.26.16.250#g' /etc/keepalived/keepalived.conf
ssh root@172.26.16.251 sed -i 's#172.26.16.249#172.26.16.251#g' /etc/keepalived/keepalived.conf
注意:不需修改172.26.16.249,只需修改其他IP即可
e、創建健康檢查腳本
[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 work]# vi /opt/check_port.sh
添加下面內容:
CHK_PORT=$1
if [ -n "$CHK_PORT" ];then
PORT_PROCESS=`ss -lt|grep $CHK_PORT|wc -l`
if [ $PORT_PROCESS -eq 0 ];then
echo "Port $CHK_PORT Is Not Used,End."
exit 1
fi
else
echo "Check Port Cant Be Empty!"
fi
f、啓動keeplived
[root@k8s-01 ~]# cd /opt/k8s/work
for NODE in k8s-01 k8s-02 k8s-03; do
echo "--- $NODE ---"
scp -r /opt/check_port.sh $NODE:/etc/keepalived/
ssh $NODE 'systemctl enable --now keepalived'
done
g、查看是否成功
[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 ~]# ping 172.26.16.252
h、檢查是否啓動成功(分別在其他服務器執行)
[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 work]# ps -ef|grep keep
如果沒有啓動,請執行下面的命令:
[root@k8s-01 ~]# systemctl start keepalived