K8s ingress資源和ingress controller

什麼是Ingress？

Ingress：就是能利用 Nginx（不常用）、Haproxy（不常用）、Traefik（常用）、Envoy（常用）等負載均衡器暴露集羣內服務的工具。Ingress提供七層負載均衡能力，可以通過 Ingress 配置提供外部可訪問的 URL、負載均衡、SSL、基於名稱的虛擬主機等。作爲集羣流量接入層，Ingress 的高可靠性顯得尤爲重要。

我們知道service的表現形式爲IP:PORT，即工作在第四層傳輸層（TCP/IP層），那麼對於不同的URL地址經常對應用不同的後端服務或者虛擬服務器，這些應用層的轉發機制僅通過kubernetes的service機制是無法實現的，這種情況我麼可以使用ingress策略定義和一個具體的ingress Controller，兩者結合實現一個完整的Ingress 負載均衡，這個負載均衡是基於nginx七層反向代理來實現的，ingress工作原理如下圖：

外部客戶端通過訪問負載均衡器，然後調度到service上，然後在調度到IngressController，IngressController通過Ingress規則（域名或虛擬主機）訪問到後端pod，而在Ingress規則當中對應的主機是又service分組來設定的，可以看到，這幅圖有2種service，最上面的service是用來對外提供服務的，而下面2個service是僅僅對後端pod分組,不被調度時使用,如果後端pod發生變動,則ingress就會將變動信息注入到,ingress controller管理的7層負載nginx的配置文件中.。

Ingress-nginx項目地址：https://github.com/kubernetes/ingress-nginx

部署Ingress控制器

下載mandatory.yaml文件

[root@s1 ~]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml

部署

[root@s1 ~]# kubectl apply -f mandatory.yaml 
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created

[root@s1 ~]# kubectl get pods -n ingress-nginx -w  -o wide
NAME                                        READY   STATUS              RESTARTS   AGE   IP       NODE   NOMINATED NODE   READINESS GATES
nginx-ingress-controller-568867bf56-6gxdx   0/1     ContainerCreating   0          59s   <none>   n1     <none>           <none>
nginx-ingress-controller-568867bf56-6gxdx   0/1     Running             0          61s   10.244.1.28   n1     <none>           <none>
nginx-ingress-controller-568867bf56-6gxdx   1/1     Running             0          65s   10.244.1.28   n1     <none>           <none>

nginx-ingress-controller部署在n1上,一個deployment控制器,一個replicaset,一個pod。

接下來還需要部署一個service-nodeport服務,才能實現把集羣外部流量接入到集羣中來:

[root@s1 ~]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/baremetal/service-nodeport.yaml

在文件中指定一下nodeport方便部署應用，修改文件中加兩個nodePort參數，併爲service設置已部署ingress的pod標籤，最終如下：

[root@s1 ~]# vim service-nodeport.yaml 
apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
      nodePort: 30080
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
      nodePort: 30443
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

[root@s1 ~]# kubectl apply -f service-nodeport.yaml 
service/ingress-nginx created
[root@s1 ~]# kubectl get svc -n ingress-nginx
NAME            TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.96.157.3   <none>        80:30080/TCP,443:30443/TCP   50s

我們直接通過n1節點的ip就可以訪問到應用：

[root@s1 ~]# curl 192.168.100.50:30080
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>openresty/1.15.8.2</center>
</body>
</html>

因爲後端沒有實際應用的80端口pod，結果自然就返回404。

接下來就部署後端service及pod：

示例一：

定義myapp service

[root@s1 ingress]# vim  deploy-demo.yaml 
apiVersion: v1
kind: Service
metadata:
  name: myapp-svc
  namespace: default
spec:
  selector:
    app: myapp
    release: canary
  clusterIP: "None"   
  ports:
  - port: 80
    targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deploy
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: myapp
      release: canary
  template:
    metadata:
      labels:
        app: myapp
        release: canary
    spec:
      containers:
      - name: myapp
        image: ikubernetes/myapp:v1
        ports:
        - name: http
          containerPort: 80

[root@s1 ingress]# kubectl apply -f deploy-demo.yaml 
service/myapp-svc created
deployment.apps/myapp-deploy created
[root@s1 ingress]# kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   12d
myapp-svc    ClusterIP   None         <none>        80/TCP    12s
[root@s1 ingress]# kubectl get pods
NAME                            READY   STATUS    RESTARTS   AGE
myapp-deploy-7d574d56c7-9p5ql   1/1     Running   0          30s
myapp-deploy-7d574d56c7-j6xkk   1/1     Running   0          30s

把myapp service通過ingress發佈出去

下面我們再定義一個清單文件，把myapp應用通過Ingress（相當於nginx的反向代理功能）發佈出去：

[root@s1 ingress]# vim ingress-myapp.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-myapp
  namespace: default #要和deployment和要發佈的service處於同一個名稱空間
  annotations:  #這個註解說明我們要用到的ingress-controller是nginx，而不是traefic，enjoy
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: myapp.lemon.com  #表示訪問這個域名，就會轉發到後端myapp管理的pod上的服務
    http:
      paths:
      - path:
        backend:
          serviceName: myapp-svc
          servicePort: 80

[root@s1 ingress]# kubectl apply -f ingress-myapp.yaml 
ingress.extensions/ingress-myapp created
[root@s1 ingress]# kubectl describe ingress
Name:             ingress-myapp
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host             Path  Backends
  ----             ----  --------
  myapp.lemon.com  
                      myapp-svc:80 (10.244.1.29:80,10.244.2.26:80)
Annotations:
  kubernetes.io/ingress.class:                       nginx
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-myapp","namespace":"default"},"spec":{"rules":[{"host":"myapp.lemon.com","http":{"paths":[{"backend":{"serviceName":"myapp-svc","servicePort":80},"path":null}]}}]}}

Events:
  Type    Reason  Age    From                      Message
  ----    ------  ----   ----                      -------
  Normal  CREATE  2m32s  nginx-ingress-controller  Ingress default/ingress-myapp
[root@s1 ingress]# kubectl get pods -n ingress-nginx
NAME                                        READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-568867bf56-6gxdx   1/1     Running   0

進入ingress-controller交互式命令行裏面，可以清晰的看到nginx是怎麼反向代理我們myapp.lemon.com的：

[root@s1 ~]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-568867bf56-6gxdx -- /bin/sh
$cat nginx.conf

外部瀏覽器中訪問，正確返回pod數據。如圖：

示例二：

將tomcat服務添加至ingress-nginx中

[root@s1 ingress]# vim tomcat-demo.yaml
apiVersion: v1
kind: Service
apiVersion: v1
kind: Service
metadata:
  name: tomcat
  namespace: default
spec:
  selector:
    app: tomcat
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  - name: ajp
    port: 8009
    targetPort: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: tomcat
  template:
    metadata:
      labels:
        app: tomcat
    spec:
      containers:
      - name: tomcat
        image: tomcat:7-alpine
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          containerPort: 8080
        - name: ajp
          containerPort: 8009

[root@s1 ingress]# kubectl apply -f tomcat-demo.yaml 
service/tomcat created
deployment.apps/tomcat created
[root@s1 ingress]# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP             12d
myapp-svc    ClusterIP   None            <none>        80/TCP              46m
tomcat       ClusterIP   10.99.236.187   <none>        8080/TCP,8009/TCP   10s
[root@s1 ingress]# kubectl get pods
NAME                            READY   STATUS    RESTARTS   AGE
myapp-deploy-7d574d56c7-9p5ql   1/1     Running   0          49m
myapp-deploy-7d574d56c7-j6xkk   1/1     Running   0          49m
tomcat-59f87c8677-66lkn         1/1     Running   0          3m22s
tomcat-59f87c8677-zvpr9         1/1     Running   0          3m22s

將tomcat服務添加至ingress-nginx中

[root@s1 ingress]# vim ingress-tomcat.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-mytomcat
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: mytomcat.lemon.com
    http:
      paths:
      - path:
        backend:
          serviceName: tomcat
          servicePort: 8080

[root@s1 ingress]# kubectl apply -f ingress-tomcat.yaml 
ingress.extensions/ingress-mytomcat created
[root@s1 ingress]# kubectl get ingress
NAME               HOSTS                ADDRESS       PORTS   AGE
ingress-myapp      myapp.lemon.com      10.96.157.3   80      37m
ingress-mytomcat   mytomcat.lemon.com   10.96.157.3   80      35s

可以看到ingress控制器已經創建。

[root@s1 ingress]# kubectl describe ingress 
Name:             ingress-myapp
Namespace:        default
Address:          10.96.157.3
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host             Path  Backends
  ----             ----  --------
  myapp.lemon.com  
                      myapp-svc:80 (10.244.1.29:80,10.244.2.26:80)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-myapp","namespace":"default"},"spec":{"rules":[{"host":"myapp.lemon.com","http":{"paths":[{"backend":{"serviceName":"myapp-svc","servicePort":80},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  CREATE  42m   nginx-ingress-controller  Ingress default/ingress-myapp
  Normal  UPDATE  41m   nginx-ingress-controller  Ingress default/ingress-myapp


Name:             ingress-mytomcat
Namespace:        default
Address:          10.96.157.3
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host                Path  Backends
  ----                ----  --------
  mytomcat.lemon.com  
                         tomcat:8080 (10.244.1.30:8080,10.244.2.27:8080)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-mytomcat","namespace":"default"},"spec":{"rules":[{"host":"mytomcat.lemon.com","http":{"paths":[{"backend":{"serviceName":"tomcat","servicePort":8080},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:
  Type    Reason  Age    From                      Message
  ----    ------  ----   ----                      -------
  Normal  CREATE  6m13s  nginx-ingress-controller  Ingress default/ingress-mytomcat
  Normal  UPDATE  5m44s  nginx-ingress-controller  Ingress default/ingress-mytomcat

訪問Tomcat資源看看：

基於ssl協議的訪問

先做個自籤的證書

[root@s1 ingress]# openssl genrsa -out tls.key 2048
Generating RSA private key, 2048 bit long modulus
.......................................+++
.......................................+++
e is 65537 (0x10001)
[root@s1 ingress]# openssl req -new -x509 -key tls.key  -out tls.crt -subj /C=CN/ST=Beijing/O=DevOps/CN=mytomcat.lemon.com
[root@s1 ingress]# ls
deploy-demo.yaml  ingress-myapp.yaml  ingress-tomcat.yaml  tls.crt  tls.key  tomcat-demo.yaml

創建secret

[root@s1 ingress]# kubectl create secret tls mytomcat-ingress-secret --cert=tls.crt --key=tls.key
secret/mytomcat-ingress-secret created

查看證書

[root@s1 ingress]# kubectl describe secret mytomcat-ingress-secret
Name:         mytomcat-ingress-secret
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  kubernetes.io/tls

Data
====
tls.crt:  1249 bytes
tls.key:  1675 bytes

通過secret把證書注入到pod中。

[root@s1 ingress]# vim ingress-tomcat-tls.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-mytomcat-tls
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - mytomcat.lemon.com  #這裏寫域名
    secretName: mytomcat-ingress-secret   #這裏寫secret證書名稱
  rules:
  - host: mytomcat.lemon.com
    http:
      paths:
      - path:
        backend:
          serviceName: tomcat
          servicePort: 8080
[root@s1 ingress]# kubectl apply -f ingress-tomcat-tls.yaml 
ingress.extensions/ingress-mytomcat-tls created

訪問https資源，pod上定義的30443端口：

K8s ingress資源和ingress controller

什麼是Ingress？

部署Ingress控制器

示例一：

示例二：

基於ssl協議的訪問

Nginx R31 doc 官方文檔-01-nginx 如何安裝

操作系統概論精簡筆記

K8s ingress資源和ingress controller

K8S Pod控制器應用進階

使用kubeadm安裝kubernetes（單master）

K8s資源清單定義入門

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結