環境
. kubernetes 1.14.3
. traefik V1.7.12
.IP 192.168.30.35
.kubectl label nodes ingress ingress=yes
https證書申請
推薦使用acme.sh 申請免費證書，具體方法不做詳細介紹
使用自簽證書
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=*.mddgame.com"traefik配置
添加traefik.toml文件：
1、http,https 同時對外提供服務
defaultEntryPoints = ["http","https"]
[entryPoints]
[entryPoints.http]
address = ":80"
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[[entryPoints.https.tls.certificates]]
certFile = "/certs/tls.crt"
keyFile = "/certs/tls.key"
2、http 強制跳轉https
defaultEntryPoints = ["http","https"]
[kubernetes]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[[entryPoints.https.tls.certificates]]
CertFile = "/certs/tls.crt"
KeyFile = "/certs/tls.key"其中tls.crt和tls.key就是證書文件，注意證書文件名必須爲固定。掛載到容器內後就會讀到該文件。
創建證書secret 方便掛在
kubectl -n kube-system create secret generic mddgame-tls-cert --from-file=tls.key --from-file=tls.crt
注意：由於secret是不能跨命名空間的，如果應用是部署在default命名空間或者其它命名空間，那還需要在default命名空間創建一個該secret，修改上面最後面的-n kube-system爲其它命名空間的名字即可。
創建configmap
kubectl create configmap traefik-conf --from-file=traefik.toml -n kube-systemtraefik部署配置
1、traefik-rbac.yaml
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik
namespace: kube-system
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses/status
verbs:
- update
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik
subjects:
- kind: ServiceAccount
name: traefik
namespace: kube-system2 創建 traefik 用戶
vi traefik-serviceaccount.yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: traefik
namespace: kube-system3、deployment 方式部署yaml
vi traefik-deployment-https.yaml
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: traefik
namespace: kube-system
labels:
k8s-app: traefik
spec:
replicas: 1
selector:
matchLabels:
k8s-app: traefik
template:
metadata:
labels:
k8s-app: traefik
name: traefik
spec:
serviceAccountName: traefik
terminationGracePeriodSeconds: 60
volumes:
- name: ssl
secret:
secretName: mddgame-tls-cert
- name: config
configMap:
name: traefik-conf
defaultMode: 0644
items:
- key: traefik.toml
path: traefik.toml
hostNetwork: true # 如果不使用hostNetwork 配置hostPort 443端口映射到宿主機會出現訪問不了k8s api 10.64.0.1:443 端口
dnsPolicy: ClusterFirstWithHostNet
containers:
- image: traefik
name: traefik
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /certs
name: "ssl"
- mountPath: /etc/traefik.toml
subPath: traefik.toml
name: "config"
ports:
- name: http
containerPort: 80
hostPort: 80
- name: https
containerPort: 443
hostPort: 443
- name: admin
containerPort: 8080
args:
- --api
- --web
- --api.dashboard
- --logLevel=INFO
- --web.metrics
- --metrics.prometheus
- --web.metrics.prometheus
- --kubernetes
- --traefiklog
- --traefiklog.format=json
- --accesslog
- --accesslog.format=json
- --accessLog.fields.headers.defaultMode=redact
- --insecureskipverify=true
- --configFile=/etc/traefik.toml
nodeSelector:
ingress: "yes"
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/ingress
operator: Equal
4、daemonset 方式部署
vi traefik-daemonset-https.yaml
---
kind: DaemonSet
apiVersion: extensions/v1beta1
metadata:
name: traefik
namespace: kube-system
labels:
k8s-app: traefik
spec:
selector:
matchLabels:
k8s-app: traefik
template:
metadata:
labels:
k8s-app: traefik
name: traefik
spec:
serviceAccountName: traefik
terminationGracePeriodSeconds: 60
volumes:
- name: ssl
secret:
secretName: mddgame-tls-cert
- name: config
configMap:
name: traefik-conf
defaultMode: 0644
items:
- key: traefik.toml
path: traefik.toml
hostNetwork: true #如果不使用hostNetwork 配置hostPort 443端口映射到宿主機會出現訪問不了k8s api 10.64.0.1:443 端口
dnsPolicy: ClusterFirstWithHostNet
containers:
- image: traefik
name: traefik
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /certs
name: "ssl"
- mountPath: /etc/traefik.toml
subPath: traefik.toml
name: "config"
ports:
- name: http
containerPort: 80
hostPort: 80
- name: https
containerPort: 443
hostPort: 443
- name: admin
containerPort: 8080
securityContext:
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
args:
- --api
- --web
- --api.dashboard
- --logLevel=INFO
- --web.metrics
- --metrics.prometheus
- --web.metrics.prometheus
- --kubernetes
- --traefiklog
- --traefiklog.format=json
- --accesslog
- --accesslog.format=json
- --accessLog.fields.headers.defaultMode=redact
- --insecureskipverify=true
- --configFile=/etc/traefik.toml
nodeSelector:
ingress: "yes"
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/ingress
operator: Equal5、創建traefik Service
vi traefik-service.yaml
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: traefik
name: traefik
namespace: kube-system
spec:
selector:
k8s-app: traefik
clusterIP: None
ports:
- protocol: TCP
port: 80
name: http
- protocol: TCP
port: 443
name: https
- protocol: TCP
port: 8080
name: admin6、創建traefik ServiceMonitor
vi traefik-serviceMonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
labels:
k8s-app: traefik
name: traefik
namespace: monitoring
spec:
endpoints:
- honorLabels: true
interval: 15s
port: admin
jobLabel: k8s-app
namespaceSelector:
matchNames:
- kube-system
selector:
matchLabels:
k8s-app: traefik7、創建 traefik dashboard Ingress
vi traefik-dashboard.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: traefik-dashboard
namespace: kube-system
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/frontend-entry-points: http,https
spec:
rules:
- host: traefik.mddgame.com
http:
paths:
- backend:
serviceName: traefik
servicePort: 8080
tls:
- secretName: mddgame-tls-cert執行yaml 創建traefik 服務
kubectl apply -f .驗證traefik 服務
kubectl get all -n kube-system | grep traefik
root@Qist:/mnt/e/work/k8s-game# kubectl get all -n kube-system | grep traefik
pod/traefik-2d5k8 1/1 Running 0 16h
service/traefik ClusterIP None <none> 80/TCP,443/TCP,8080/TCP 16h
daemonset.apps/traefik 1 1 1 1 1 ingress=yes 16h
寫hosts
192.168.30.35 traefik.mddgame.com
訪問traefik.mddgame.comhttp 訪問
https 訪問
創建測試
vi nginx.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 2
selector:
matchLabels:
k8s-app: nginx
template:
metadata:
labels:
k8s-app: nginx
spec:
containers:
- name: nginx
image: docker.mddgame.com/library/nginx
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
name: web
protocol: TCP
- containerPort: 8080
name: vts
protocol: TCP
readinessProbe:
failureThreshold: 10
httpGet:
path: /healthz
port: vts
scheme: HTTP
initialDelaySeconds: 3
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 3
resources:
requests:
cpu: 200m
memory: 200Mi
- name: nginx-vts-exporter
image: docker.mddgame.com/library/nginx-vts-exporter
imagePullPolicy: IfNotPresent
args:
- "-nginx.scrape_uri=http://localhost:8080/format/json"
ports:
- containerPort: 9913
name: http-metrics
protocol: TCP
resources:
requests:
memory: 30Mi
cpu: 102m
limits:
memory: 50Mi
cpu: 250m
---
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: nginx
name: nginx
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/affinity: "true" # 後端基於Cookie會話
traefik.ingress.kubernetes.io/load-balancer-method: drr #修改負載方式
spec:
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 10800
selector:
k8s-app: nginx
ports:
- protocol: TCP
port: 80
name: web
- protocol: TCP
port: 8080
name: vts
- protocol: TCP
port: 9913
name: http-metrics
type: ClusterIP
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx
annotations:
kubernetes.io/ingress.class: traefik
traefik.frontend.rule.type: PathPrefixStrip
traefik.ingress.kubernetes.io/frontend-entry-points: http,https
spec:
rules:
- host: nginx.mddgame.com
http:
paths:
- path: /
backend:
serviceName: nginx
servicePort: 80
tls:
- secretName: mddgame-tls-cert
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
labels:
k8s-app: nginx
name: nginx
spec:
endpoints:
- honorLabels: true
interval: 15s
port: http-metrics
jobLabel: k8s-app
selector:
matchLabels:
k8s-app: nginx
創建nginx 服務
kubectl apply -f .
hosts 寫入
192.168.30.35 nginx.mddgame.com 
正常打開
外部業務使用traefik 對外提供服務
以Confluence 爲例
vi wiki.yaml
apiVersion: v1
kind: Service
metadata:
labels:
k8s-app: wiki
name: wiki
namespace: default
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/affinity: "true"
traefik.ingress.kubernetes.io/load-balancer-method: drr
spec:
clusterIP: None
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
sessionAffinity: None
type: ClusterIP
---
apiVersion: v1
kind: Endpoints
metadata:
labels:
k8s-app: wiki
name: wiki
namespace: default
subsets:
- addresses:
- ip: 192.168.30.11 # 多個ip 可以直接在下一行添加 -ip:192.168.30.22
ports:
- name: http
port: 8080
protocol: TCP
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: wiki
namespace: default
annotations:
kubernetes.io/ingress.class: traefik
traefik.frontend.rule.type: PathPrefixStrip
traefik.ingress.kubernetes.io/frontend-entry-points: http,https
traefik.ingress.kubernetes.io/redirect-entry-point: https # 強制調整到https
spec:
rules:
- host: wiki.mddgame.com
http:
paths:
- path: /
backend:
serviceName: wiki
servicePort: 8080
tls:
- secretName: mddgame-tls-cert綁定hosts
192.168.30.35 wiki.mddgame.com
驗證是否能打開
能夠正常打開