17.Traefik-ingress

官方文檔 https://docs.traefik.cn/

Ingress簡介

對於基於HTTP的服務來說，不同的URL地址經常對應到不同的後端服務或者虛擬服務器，這些應用層的轉發機制僅通過Kubernetes的Service機制是無法實現的。從Kubernetes v1.1版本開始新增Ingress資源對象，用於將不同URL訪問請求轉發到後端不同的Service，以實現HTTP層的業務路由機制。Kubernetes使用一個Ingress策略定義和一個具體的Ingress Controller，兩者結合並實現一個完整的Ingress負載均衡器。
使用Ingress進行負載分發時，Ingress Controller通過api-server監聽ingress和service的變化，將基於Ingress策略將客戶端請求直接轉發到Service對應的後端Endpoint上，這樣會跳過kube-proxy的轉發功能。
- 未配置ingress:
集羣外部 --> NodePort --> k8s Service

• 配置ingress:
  集羣外部 --> Ingress --> k8s Service
  Ingress Controller 除了traefik，還有Haproxy、Linkerd、nginx等其他類型。

Traefik 簡介

Traefik 是一個爲了讓部署微服務更加便捷而誕生的現代HTTP反向代理、負載均衡工具。它支持多種後臺（Docker、Swarm、Kubernetes、Mesos、Consul、Etcd、zookeeper等）來自動化、動態的應用他的配置文件來設置。



特性

• 它非常快
• 它無需安裝其他依賴，通過Go語音編寫的單一可執行文件
• 支持Rest API
• 支持多種後臺
• 後臺監控，可以監聽後臺變化而自動化應用新的配置文件設置
• 配置文件熱更新，無需重啓進程
• 正常結束http連接
• 後端斷路器
• 輪詢，rebalancer負載均衡
• Rest Metrics
• 支持最小化官方Docker鏡像
• 後臺支持SSL
• 前臺支持SSL
• 清爽的AngularJS前端頁面
• 支持WebSocket
• 支持HTTP/2
• 網絡錯誤重試
• 支持Let’s Encrypt（自動更新https證書）
• 高可用集羣模式

部署traefik

Traefik可以監聽你的服務發現、管理API，並且每當你的的微服務被添加、移除、殺死或者更新都會被感知，並且可以自動生成它們的配置文件。指向到你服務的路由將會被直接創建出來。

安裝traefik ingress-controller

因爲Traefik需要監聽服務發現和管理API，所以需要對Ingress做RBAC授權。
kubecl apply -f traefik-rbac.yaml

# traefik-rbac.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - pods
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system

創建ConfigMap 將域名ssl證書掛載到traefik中，將各個域名的SSL證書放到/data/srv/traefik/ssl/目錄下。

# kubectl -n kube-system create configmap traefik-ssl --from-file=/data/srv/traefik/ssl/

通過Daemonset和Service將traefik部署到集羣的所有節點。 Traefik Pod中80端口爲traefik ingress-controller的服務端口，8080端口爲traefik的管理WEB界面；爲後續配置方便指定80端口暴露NodePort端口爲23456，443端口暴露NodePort端口爲23457。
通過ConfigMap的形式定義traefik的配置文件traefik.toml，加載各域名SSL證書。
kubectl apply -f traefik-controller.yaml

# traefik-controller.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: traefik-conf
  namespace: kube-system
data:
  traefik.toml: |
    # 設置insecureSkipVerify = true，可以配置backend爲443(比如dashboard)的ingress規則
    insecureSkipVerify = true
    defaultEntryPoints = ["http", "https"]
    [entryPoints]
      [entryPoints.http]
        address = ":80"
        ### 配置http 強制跳轉 https
        [entryPoints.http.redirect]
          entryPoint = "https"
        ### 配置只信任trustedIPs傳遞過來X-Forwarded-*，默認全部信任；爲了防止客戶端地址僞造，需開啓這個
        #[entryPoints.http.forwardedHeaders]
        #  trustedIPs = ["10.1.0.0/16", "172.20.0.0/16", "192.168.1.3"]
      [entryPoints.https]
        address = ":443"
        [entryPoints.https.tls]
          [[entryPoints.https.tls.certificates]]
            CertFile = "/ssl/baijiahulian.pem"
            KeyFile = "/ssl/baijiahulian.key"
          [[entryPoints.https.tls.certificates]]
            CertFile = "/ssl/weishi100.pem"
            KeyFile = "/ssl/weishi100.key"
          [[entryPoints.https.tls.certificates]]
            CertFile = "/ssl/gaotu100.pem"
            KeyFile = "/ssl/gaotu100.key"
          [[entryPoints.https.tls.certificates]]
            CertFile = "/ssl/genshuixue.crt"
            KeyFile = "/ssl/genshuixue.key"
          [[entryPoints.https.tls.certificates]]
            CertFile = "/ssl/umeng100.pem"
            KeyFile = "/ssl/umeng100.key"
          [[entryPoints.https.tls.certificates]]
            CertFile = "/ssl/wenzaizhibo.crt"
            KeyFile = "/ssl/wenzaizhibo.key"
          [[entryPoints.https.tls.certificates]]
            CertFile = "/ssl/babyabc100.pem"
            KeyFile = "/ssl/babyabc100.key"
          [[entryPoints.https.tls.certificates]]
            CertFile = "/ssl/sodareading.pem"
            KeyFile = "/ssl/sodareading.key"
---
kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      volumes:
      - name: ssl
        configMap:
          name: traefik-ssl
      - name: config
        configMap:
          name: traefik-conf
      #nodeSelector:
      #  node-role.kubernetes.io/traefik: "true"
      containers:
      - image: traefik:v1.7.4
        imagePullPolicy: IfNotPresent
        name: traefik-ingress-lb
        volumeMounts:
        - mountPath: "/ssl"
          name: "ssl"
        - mountPath: "/config"
          name: "config"
        resources:
          limits:
            cpu: 1000m
            memory: 800Mi
          requests:
            cpu: 500m
            memory: 600Mi
        args:
        - --configfile=/config/traefik.toml
        - --api
        - --kubernetes
        - --logLevel=INFO
        securityContext:
          capabilities:
            drop:
              - ALL
            add:
              - NET_BIND_SERVICE
        ports:
          - name: http
            containerPort: 80
            hostPort: 80
          - name: https
            containerPort: 443
            hostPort: 443
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      # 該端口爲 traefik ingress-controller的服務端口
      port: 80
      # 集羣hosts文件中設置的 NODE_PORT_RANGE 作爲 NodePort的可用範圍
      # 從默認20000~60000之間選一個可用端口，讓ingress-controller暴露給外部的訪問
      nodePort: 23456
      name: http
    - protocol: TCP
      #
      port: 443
      nodePort: 23457
      name: https
    - protocol: TCP
      # 該端口爲 traefik 的管理WEB界面
      port: 8080
      name: admin
  type: NodePort

驗證traefik ingress-controller

[root@of-bj-k8s-deploy01 tls]# kubectl get daemonset -n kube-system traefik-ingress-controller
NAME                         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
traefik-ingress-controller   5         5         5       5            5           <none>          45h
[root@of-bj-k8s-deploy01 tls]# kubectl get svc -n kube-system traefik-ingress-service
NAME                      TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)                                     AGE
traefik-ingress-service   NodePort   10.0.210.214   <none>        80:23456/TCP,443:23457/TCP,8080:21325/TCP   45h
[root@of-bj-k8s-deploy01 tls]#

創建https ingress 例子

創建測試用例

[root@of-bj-k8s-deploy01 tls]# kubectl run test-hello --image=nginx --port=80 --expose
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
service/test-hello created
deployment.apps/test-hello created
[root@of-bj-k8s-deploy01 tls]# kubectl get all
NAME                              READY   STATUS    RESTARTS   AGE
pod/test-hello-86f5bff8b4-w6xqh   1/1     Running   0          36s

NAME                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
service/test-hello   ClusterIP   10.0.200.209   <none>        80/TCP    36s

NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/test-hello   1/1     1            1           36s

NAME                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/test-hello-86f5bff8b4   1         1         1       36s

在default命名空間創建對應的secret，併爲測試用例配置https路由策略。

[root@of-bj-k8s-deploy01 tls]# kubectl create secret tls baijiahulian-cert --key=/data/srv/traefik/ssl/baijiahulian.key --cert=/data/srv/traefik/ssl/baijiahulian.pem
secret/baijiahulian-cert created
[root@of-bj-k8s-deploy01 tls]# cat hello-tls.ing.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: hello-tls-ingress
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: hello.baijiahulian.com
    http:
      paths:
      - backend:
          serviceName: test-hello
          servicePort: 80
  tls:
  - secretName: baijiahulian-cert
[root@of-bj-k8s-deploy01 tls]# kubectl apply -f hello-tls.ing.yaml
ingress.extensions/hello-tls-ingress created
[root@of-bj-k8s-deploy01 tls]#

創建負載均衡器

在雲平臺上配置負載均衡器，監聽80端口對應集羣node節點上的23456端口，監聽443端口對應集羣node節點上的23457端口。
在客戶端hosts增加記錄$SLB_IP hello.baijiahulian.com之後，可以在瀏覽器驗證訪問。

$ curl -w %{http_code} https://hello.baijiahulian.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
200

爲Traefik WEB管理頁面配置ingress規則

[root@of-bj-k8s-deploy01 traefik]# cat traefik-ui.ing.yaml
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  rules:
  - host: traefik-ui.baijiahulian.com
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-ingress-service
          servicePort: 8080
[root@of-bj-k8s-deploy01 traefik]# kubectl apply -f traefik-ui.ing.yaml
ingress.extensions/traefik-web-ui created

在客戶端增加記錄$SLB_IP traefik-ui.baijiahulian.com，並在瀏覽器進行驗證訪問。


Traefik 配置基礎及全局配置文件

Traefik 官網上已經對Traefik的概念及用法已經做了全面的介紹，本文不在進行具體的描述和配置驗證。