Kubernetes系列之：通過hostNetwork、NodePort、Ingress 訪問K8s應用

本篇重點講解K8s中訪問應用的幾種方法，我將在k8s中布屬一個nginx示例，示例將通過：hostNetwork，NodePort，Ingress方式來訪問。

示例一：通過hostNetwork，hostPort

布屬Nginx，創建nginx.yaml文件，注意：在yaml中加的hostNetwork: true

[root@k8s-0001 nginx]# vi nginx.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx-test
  labels:
    name: nginx-test
  namespace: develop
spec:
  replicas: 1
  selector:
    name: nginx-test
  template:
    metadata:
      labels:
       name: nginx-test
      namespace: develop
    spec:
      hostNetwork: true
      containers:
      - name: nginx-test
        image: docker.io/nginx
        ports:
        - containerPort: 80

查看布屬後應用，可以看到nginx-test的IP是宿主機的內網IP，而testapi的是虛擬IP，這就是加不加hostNetwork的區別

[root@k8s-0001 nginx]# kubectl get pods -n develop -o wide
NAME                                        READY   STATUS    RESTARTS   AGE     IP              NODE       NOMINATED NODE   READINESS GATES
nginx-test-ms852                            1/1     Running   0          4m54s   192.168.1.85    k8s-0003   <none>           <none>
testapi-85664b498-vl749                     1/1     Running   3          11d     10.244.2.42     k8s-0002   <none>           <none>
testapi-85664b498-zxqml                     1/1     Running   11         11d     10.244.3.47     k8s-0004   <none>           <none>

測試：布屬採用的是80端口，所以直接通過curl -i 192.168.1.85 來測試

[root@k8s-0001 nginx]# curl -i 192.168.1.85
HTTP/1.1 200 OK
Server: nginx/1.17.8
Date: Sat, 15 Feb 2020 06:06:15 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 21 Jan 2020 13:36:08 GMT
Connection: keep-alive
ETag: "5e26fe48-264"
Accept-Ranges: bytes

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

示例二：通過NodePort訪問

增加Services：K8s中實現的服務發現，Services通過NodePort來向外部暴露端口

apiVersion: v1
kind: Service
metadata:
  name: nginx-test-svc
  labels:
   name: nginx-test-svc
  namespace: develop
spec:
  type: NodePort
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
    name: http
  selector:
    name: nginx-test

[root@k8s-0001 nginx]# kubectl get svc -n develop  -o wide
NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE     SELECTOR
nginx-test-svc             NodePort    10.102.30.135    <none>        80:32134/TCP                 9m38s   name=nginx-test

我們看到PORT（S）爲80:32134 ，意思是該service對外暴露的端口是 32134並且該端口映身到了應用的80端口。當然，也可以通過nodePort屬性來指定一個固定的端口，如下：

apiVersion: v1
kind: Service
metadata:
  name: nginx-test-svc
  labels:
   name: nginx-test-svc
  namespace: develop
spec:
  type: NodePort
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
    nodePort: 30011
    name: http
  selector:
    name: nginx-test

[root@k8s-0001 nginx]# kubectl get svc -n develop  -o wide
NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE     SELECTOR
nginx-test-svc             NodePort    10.102.30.135    <none>        80:30011/TCP                 9m38s   name=nginx-test

測試：service是位爲master上，IP爲：114.119.181.149 ，所以我們可以通過114.119.181.149:30011來訪問。

示例三：通過Ingress訪問

通過Ingress來訪問應用，我們需要先搞清楚邏輯關係，我在搭建的時候走了一些彎路，是有些與想的不一樣導致。

Ingress主要由三個部份組成：Ingress-service、Ingress-Controller、Ingress。

通過Ingress訪問的邏輯爲：Client Request -> Ingress-service(通過NodePort暴露端口) -> Ingress -> Service -> Deployment

開始布屬，Ingress的yaml可以從以下鏈接獲取最新，我們需要根據具體的情況，做一些修改。

https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/

https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.20.0/deploy/mandatory.yaml

ingress-controller.yaml，修改的點主要爲Namespace，因爲我的應用布屬在develop這個名稱空間下。

apiVersion: v1
kind: Namespace
metadata:
  name: develop

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    app.kubernetes.io/name: default-http-backend
    app.kubernetes.io/part-of: ingress-nginx
  namespace: develop
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: default-http-backend
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: default-http-backend
        app.kubernetes.io/part-of: ingress-nginx
    spec:
      terminationGracePeriodSeconds: 60
      containers:
        - name: default-http-backend
          # Any image is permissible as long as:
          # 1. It serves a 404 page at /
          # 2. It serves 200 on a /healthz endpoint
          image: k8s.gcr.io/defaultbackend-amd64:1.5
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            timeoutSeconds: 5
          ports:
            - containerPort: 8080
          resources:
            limits:
              cpu: 10m
              memory: 20Mi
            requests:
              cpu: 10m
              memory: 20Mi

---
apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
  namespace: develop
  labels:
    app.kubernetes.io/name: default-http-backend
    app.kubernetes.io/part-of: ingress-nginx
spec:
  ports:
    - port: 80
      targetPort: 8080
  selector:
    app.kubernetes.io/name: default-http-backend
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: develop
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: develop
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: develop
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: develop
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: develop
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: develop
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: develop

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: develop

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: develop
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.20.0
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 33
            runAsUser: 33
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
            - name: https
              containerPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1

---

ingress.yaml

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: nginx-web
  namespace: develop
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: findo.hixiu.com
    http:
      paths:
      - path:
        backend:
          serviceName: findo-svc
          servicePort: web
  - host: gofindo.hixiu.com
    http:
      paths:
      - path:
        backend:
          serviceName: go-findo-svc
          servicePort: web

ingress-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: nginx-ingress-controller
  namespace: develop
spec:
  type: NodePort
  ports: 
    - name: http
      port: 80
    - name: https
      port: 443
  selector:
     app.kubernetes.io/name: ingress-nginx

測試：通過 kubectl get svc -n develop -o wide 查看已布屬的服務，可以發現nginx-ingress-controller 開發的端口爲:30033


NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE   SELECTOR
default-http-backend       ClusterIP   10.102.58.113    <none>        80/TCP                       9d    app.kubernetes.io/name=default-http-backend,app.kubernetes.io/part-of=ingress-nginx
findo-svc                  ClusterIP   10.102.47.86     <none>        6660/TCP                     9d    webapi=findo-api
go-findo-svc               ClusterIP   10.111.160.126   <none>        8080/TCP                     19h   webapi=go-findo-api
nginx-ingress-controller   NodePort    10.101.0.193     <none>        80:30033/TCP,443:30241/TCP   9d    app.kubernetes.io/name=ingress-nginx
testapi                    ClusterIP   10.108.208.200   <none>        8889/TCP                     9d    webapi=testwebapi

從ingress.yaml配置中不難看出我配置了兩個字域名，分別訪問不同的服務：go-findo-svc與findo-svc，對應的域名爲：gofindo.hixiu.com與findo.hixiu.com，這兩個服務分別是用Go與C#開發的，返回的類型不致，go返回的是json，C#返回的是text，我們來測試一下。