OpenShift 4 Hands-on Lab (12) 通過 KubeFed 配置 OpenShift 聯邦集羣

原創 dawnsky.liu 2020-06-22 09:39

文章目錄

Kubernetes Federation

Kubernetes聯邦集羣是由https://github.com/kubernetes-sigs主導開發，目前已經進入Kubefed v2的0.2.0-alpha.1版本。通過OpenShift內置的Kubefed Operator，我們可以在OpenShift之上運行聯邦集羣。在Kubefed中通過Controller Manager維護聯邦集羣中每個Kubernetes集羣成員的資源狀態。
在這裏插入圖片描述
Kubenetes Federation可以多個Kubernetes集羣之間實現以下高可用功能：

  • 簡化管理多個集羣的Kubernetes 組件(如Deployment, Service 等)。
  • 在多個集羣之間分散工作負載(容器)，以提升應用(服務)的可靠性。
  • 跨集羣的資源編排，依據編排策略在多個集羣進行應用(服務)部署。
  • 在不同集羣中，能更快速更容易地遷移應用(服務)。
  • 跨集羣的服務發現，服務可以提供給當地存取，以降低延遲。
  • 實踐多雲(Multi-cloud)或混合雲(Hybird Cloud)的部署。

準備環境

OpenShift集羣環境

我們需要2個基於4.x的OpenShift集羣環境：cluster1和cluster2，其中在cluster1部署KubeFed（即Federation Manager）。

配置客戶端環境

我們用Linux作爲客戶端運行環境，需要在客戶端上配置2個集羣的context，並且安裝kubefedctl客戶端。

  1. 在客戶端登錄OpenShift集羣cluster1和cluster2。
$ oc login <OCP_CLUSER1>
$ oc config rename-context $(oc config current-context) cluster1
$ oc login <OCP_CLUSER2>
$ oc config rename-context $(oc config current-context) cluster2
  1. 先查看將客戶端的Config Context，然後把客戶端登錄的兩個Context Name分別設爲cluster1和cluster1。（可參照《OpenShift 4 之 GitOps（4）用ArgoCD向Multi-Cluster發佈應用》）。
  2. 設置完後先將缺省Config設置到cluster1，然後再查看Config Context爲以下cluster1和cluster2。
$ oc config use-context cluster1
$ oc config get-contexts
CURRENT   NAME       CLUSTER                                                              AUTHINFO                                                                         NAM            ESPACE
*         cluster1   api-cluster-beijing-c3c2-beijing-c3c2-example-opentlc-com:6443       opentlc-mgr/api-cluster-beijing-c3c2-beijing-c3c2-example-opentlc-com:6443       def            ault
          cluster2   api-cluster-beijing-3beb-beijing-3beb-sandbox1785-opentlc-com:6443   opentlc-mgr/api-cluster-beijing-3beb-beijing-3beb-sandbox1785-opentlc-com:6443   def            ault
  1. 安裝kubefedctl客戶端
$ curl -LO https://github.com/kubernetes-sigs/kubefed/releases/download/v0.1.0-rc6/kubefedctl-0.1.0-rc6-linux-amd64.tgz
$ tar -xvf kubefedctl-0.1.0-rc6-linux-amd64.tgz
$ sudo mv kubefedctl /usr/local/bin/

安裝配置KubeFed Operator

  1. 在cluster1上創建運行KubeFed Operator的Namespace。說明：KubeFed缺省使用名爲“kube-federation-system”的項目作爲其運行環境。因此在缺省的時候，本文以下部分中的“–kubefed-namespace kube-federation-system”其實都可以省掉。如果KubeFed運行在其它項目名中，則以下相關命令都需要提供“–kubefed-namespace”參數。
$ oc --context=cluster1 create ns kube-federation-system
  1. 用集羣管理員進入OpenShift Console的Administrator視圖中的OperatorHub（確認此時是在kube-federation-system項目中）。然後找到“Kubefed Operator”點擊進入。
  2. 點擊Install，然後選擇部署在kube-federation-system項目中，最後點擊Subscribe。
  3. 在Installed Operators中等到狀態變爲“Up to date”後進入該Kubefed Operator。
    在這裏插入圖片描述
  4. 查看當前運行的Pod。
$ oc --context=cluster1 -n kube-federation-system get pods
NAME                                READY   STATUS    RESTARTS   AGE
kubefed-operator-76b674b9fd-dsjkl   1/1     Running   0          88s
  1. 在Kubefed Operator的Overview頁面中找到KubeFedWebHook，點擊Create Instance，然後接受缺省配置創建KubeFedWebHook。最後查看Pod的狀態應該如下。
$ oc --context=cluster1 -n kube-federation-system get pods
NAME                                		READY   STATUS    RESTARTS   AGE
kubefed-operator-76b674b9fd-dsjkl   		1/1     Running   0          88s
kubefed-admission-webhook-89fbcbb74-dvdq6   1/1     Running   1          10h
  1. 在Kubefed Operator的Overview頁面中找到KubeFed，點擊Create Instance，然後接受缺省配置創建KubeFed。最後查看Pod的狀態應該如下。
$ oc --context=cluster1 -n kube-federation-system get pods
NAME                                          READY   STATUS    RESTARTS   AGE
kubefed-admission-webhook-89fbcbb74-dvdq6     1/1     Running   1          10h
kubefed-controller-manager-6d5f46d745-nd8fw   1/1     Running   1          10h
kubefed-controller-manager-6d5f46d745-p6f6b   1/1     Running   1          10h
kubefed-operator-76b674b9fd-dsjkl   		  1/1     Running   0          88s
  1. 執行命令（也可在下圖中的Kubefed Opeator中的All Instance中查看），查看聯邦集羣中的對象實例。
$ oc --context=cluster1 -n kube-federation-system get kubefedconfig 
NAME      AGE
kubefed   31h
$ oc --context=cluster1 -n kube-federation-system get kubefedwebhook
NAME                      AGE
kubefedwebhook-resource   31h
$ oc --context=cluster1 -n kube-federation-system get kubefed
NAME               AGE
kubefed-resource   31h

在這裏插入圖片描述

將OpenShift集羣加入到聯邦集羣

  1. 執行以下命令，將cluster1和cluster2加入到聯邦集羣(用cluster1作爲主管)中。
$ kubefedctl join cluster1 --cluster-context cluster1 --host-cluster-context cluster1 --kubefed-namespace=kube-federation-system --v=2
I0313 02:09:12.434998    1289 join.go:159] Args and flags: name cluster1, host: cluster1, host-system-namespace: kube-federation-system, kubeconfig: , cluster-context: cluster1, secret-name: , dry-run: false
I0313 02:09:12.535522    1289 join.go:240] Performing preflight checks.
I0313 02:09:12.539877    1289 join.go:246] Creating kube-federation-system namespace in joining cluster
I0313 02:09:12.544239    1289 join.go:382] Already existing kube-federation-system namespace
I0313 02:09:12.544257    1289 join.go:254] Created kube-federation-system namespace in joining cluster
I0313 02:09:12.544267    1289 join.go:403] Creating service account in joining cluster: cluster1
I0313 02:09:12.553436    1289 join.go:413] Created service account: cluster1-cluster1 in joining cluster: cluster1
I0313 02:09:12.553451    1289 join.go:441] Creating cluster role and binding for service account: cluster1-cluster1 in joining cluster: cluster1
I0313 02:09:12.577425    1289 join.go:450] Created cluster role and binding for service account: cluster1-cluster1 in joining cluster: cluster1
I0313 02:09:12.577455    1289 join.go:809] Creating cluster credentials secret in host cluster
I0313 02:09:13.596025    1289 join.go:835] Using secret named: cluster1-cluster1-token-8dxrm
I0313 02:09:13.608147    1289 join.go:878] Created secret in host cluster named: cluster1-9d4j7
I0313 02:09:13.638964    1289 join.go:282] Created federated cluster resource
 
$ kubefedctl join cluster2 --cluster-context cluster2 --host-cluster-context cluster1 --kubefed-namespace=kube-federation-system --v=2
I0315 08:54:52.816368     799 join.go:159] Args and flags: name cluster2, host: cluster1, host-system-namespace: kube-federation-system, kubeconfig: , cluster-context: cluster2, secret-name: , dry-run: false
I0315 08:54:53.178787     799 join.go:238] Performing preflight checks.
I0315 08:54:53.411417     799 join.go:244] Creating kube-federation-system namespace in joining cluster
I0315 08:54:53.665019     799 join.go:252] Created kube-federation-system namespace in joining cluster
I0315 08:54:53.665060     799 join.go:398] Creating service account in joining cluster: cluster2
I0315 08:54:53.854303     799 join.go:408] Created service account: cluster2-cluster1 in joining cluster: cluster2
I0315 08:54:53.854334     799 join.go:436] Creating cluster role and binding for service account: cluster2-cluster1 in joining cluster: cluster2
I0315 08:54:54.413824     799 join.go:445] Created cluster role and binding for service account: cluster2-cluster1 in joining cluster: cluster2
I0315 08:54:54.413852     799 join.go:804] Creating cluster credentials secret in host cluster
I0315 08:54:54.743515     799 join.go:830] Using secret named: cluster2-cluster1-token-jhprj
I0315 08:54:54.754980     799 join.go:873] Created secret in host cluster named: cluster2-jpsq9
I0315 08:54:54.782983     799 join.go:280] Created federated cluster resource
  1. (可選步驟)-執行命令可將cluster2從聯邦集羣中去掉。
$ kubefedctl unjoin cluster2 --cluster-context cluster2 --host-cluster-context cluster1 --kubefed-namespace=kube-federation-system --v=2
  1. 查看聯邦集羣中的OpenShift集羣都是READY=True狀態。
$ oc --context=cluster1 -n kube-federation-system get kubefedclusters 
NAME       READY   AGE
cluster1   True    1m
cluster2   True    1m
  1. 執行命令設置允許運行聯邦對象類型。任何Kubernetes API支持的類型都可以成爲對應的聯邦類型（包括CRD類型），例如以下的FederatedDeployment、FederatedService。注意：這裏只是允許部署這些聯邦對象類型，而不是在聯邦集羣中對這些聯邦對象執行復制操作。
for type in namespaces secrets serviceaccounts services configmaps deployments.apps scc
do
  kubefedctl enable $type --kubefed-namespace kube-federation-system
done

執行後會有以下輸出，說明這些類型都已經註冊到聯邦集羣的FederatedTypeConfig中了。

customresourcedefinition.apiextensions.k8s.io/federatednamespaces.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/namespaces updated in namespace kube-federation-system
customresourcedefinition.apiextensions.k8s.io/federatedsecrets.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/secrets updated in namespace kube-federation-system
customresourcedefinition.apiextensions.k8s.io/federatedserviceaccounts.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/serviceaccounts created in namespace kube-federation-system
customresourcedefinition.apiextensions.k8s.io/federatedservices.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/services created in namespace kube-federation-system
customresourcedefinition.apiextensions.k8s.io/federatedconfigmaps.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/configmaps created in namespace kube-federation-system
customresourcedefinition.apiextensions.k8s.io/federateddeployments.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/deployments.apps created in namespace kube-federation-system
customresourcedefinition.apiextensions.k8s.io/federatedsecuritycontextconstraints.types.kubefed.io created
federatedtypeconfig.core.kubefed.io/securitycontextconstraints.security.openshift.io created in namespace kube-federation-system
  1. 查看聯邦集羣中FederatedTypeConfigAPI包括的API類型，結果對應上一步設置的聯邦集羣允許的API資源對象類型。
$ oc --context=cluster1 -n kube-federation-system get FederatedTypeConfig
NAME                                               AGE
configmaps                                         3m47s
deployments.apps                                   3m46s
namespaces                                         6m28s
secrets                                            5m56s
securitycontextconstraints.security.openshift.io   4m3s
serviceaccounts                                    3m48s
services                                           3m48s
  1. （可選）：可以用以下命令diable一種Kubernetes API的類型。例如下面命令會從FederatedTypeConfig中將Deployment類型刪除（並同時刪除構成其聯邦部署能力的相關CRD對象），使其不再具有在聯邦集羣中的傳播複製能力。
$ kubefedctl disable deployments.apps --delete-crd
  1. （可選）*：以下命令可以查看集羣支持的所有Kubernetes API類型。
$ oc api-resources

向聯邦集羣部署應用資源

概念說明

kubefedctl的federate命令是用來根據Kubernetes API Tpye對象生成對應的Federated化對象。其中Federated化對象中template部分就是對應一般的Kubernetes API Tpye對象。

kubefedctl federate <target kubernetes API type> <target resource name> [flags]

以下每個“kubefedctl federate”命令都將一個特定的Kubernetes API類型Federated化，即生成對應的Federated<TYPE>對象。

kubefedctl federate namespace test-namespace --kubefed-namespace kube-federation-system
kubefedctl federate secrets test-secret -n test-namespace --kubefed-namespace kube-federation-system
kubefedctl federate serviceaccounts test-serviceaccount -n test-namespace --kubefed-namespace kube-federation-system
kubefedctl federate services test-service -n test-namespace --kubefed-namespace kube-federation-system
kubefedctl federate configmaps test-configmap -n test-namespace --kubefed-namespace kube-federation-system
kubefedctl federate deployments.apps test-deployment -n test-namespace --kubefed-namespace kube-federation-system
kubefedctl federate federatedsecuritycontextconstraints test-anyuid -n test-namespace --kubefed-namespace kube-federation-system

可以用“kubefedctl federate”命令的“–output string”參數將Federated化對象轉爲YAML格式輸出。

操作

參照以下操作指導可實現在集羣聯邦中部署應用。

  1. 先確認cluster1和cluster2上都沒有test-namespace項目。
$ oc --context=cluster1 get project test-namespace
Error from server (NotFound): namespaces "test-namespace" not found
$ oc --context=cluster2 get project test-namespace
Error from server (NotFound): namespaces "test-namespace" not found
  1. 在cluster1上創建test-namespace項目，然後確認cluster2還沒有test-namespace項目。這是由於test-namespace項目還未加入到KubeFed中。
$ oc --context=cluster1 new-project test-namespace
$ oc --context=cluster1 get project test-namespace   
NAME             DISPLAY NAME   STATUS
test-namespace                  Active
$ oc --context=cluster2 get project test-namespace   
Error from server (NotFound): namespaces "test-namespace" not found
  1. 執行命令，把名爲“test-namespace”的namespace對象Federated化，既生成了一個名爲“test-namespace”的federatednamespace對象。然後查看這個對象。
    說明：只有Federated化資源對象（例如下面的test-namespace項目）才能被Kubefed的Controller Manager複製到聯邦集羣的Member Cluster上。
$ kubefedctl federate namespace test-namespace --kubefed-namespace kube-federation-system
W0315 13:00:01.270320    1080 federate.go:410] Annotations defined for Namespace "test-namespace" will not appear in the template of the federated resource: map[openshift.io/sa.scc.mcs:s0:c23,c7 openshift.io/sa.scc.supplemental-groups:1000520000/10000 openshift.io/sa.scc.uid-range:1000520000/10000]
I0315 13:00:01.270486    1080 federate.go:474] Resource to federate is a namespace. Given namespace will itself be the container for the federated namespace
I0315 13:00:01.394914    1080 federate.go:503] Successfully created FederatedNamespace "test-namespace/test-namespace" from Namespace

說明：如果一個namespace中已經有資源了，可以使用“–contents”參數將項目中的已有的資源類型自動Federated化。

kubefedctl federate namespace my-namespace --contents --kubefed-namespace kube-federation-system
  1. 分別執行以下命令，查看上一步從名爲test-namespace的namespace生成的同名聯邦對象federatednamespace實例和器詳細信息。可以從詳細信息中看到federatednamespace的傳播範圍其是cluster1和cluster。
$ oc --context=cluster1 -n test-namespace get federatednamespace
NAME             AGE
test-namespace   3m
 
$ oc --context=cluster1 -n test-namespace get federatednamespace test-namespace -o yaml
apiVersion: types.kubefed.io/v1beta1
kind: FederatedNamespace
metadata:
  creationTimestamp: "2020-03-20T16:43:32Z"
  finalizers:
  - kubefed.io/sync-controller
  generation: 1
  name: test-namespace
  namespace: test-namespace
  resourceVersion: "1126829"
  selfLink: /apis/types.kubefed.io/v1beta1/namespaces/test-namespace/federatednamespaces/test-namespace
  uid: ef74af24-6ac9-11ea-92df-1222d546a5b5
spec:
  placement:
    clusterSelector:
      matchLabels: {}
  template:
    spec: {}
status:
  clusters:
  - name: cluster2
  - name: cluster1
  conditions:
  - lastTransitionTime: "2020-03-21T09:15:03Z"
    lastUpdateTime: "2020-03-21T09:15:03Z"
    status: "True"
    type: Propagation
  1. 確認此時cluster2上也有了test-namespace項目。
$ oc --context=cluster2 get project test-namespace   
NAME              DISPLAY NAME   STATUS
test-namespace                   Active
  1. 將項目名改爲test-namespace1，然後重複以上（1）-（5）步驟，確認test-namespace1項目可以自動從cluster1複製到cluster2。
  2. 下載測試應用項目。注意：以下地址的上游項目中sample-app目錄裏所有文件內容的apiversion命名空間已經不適合OpenShift 4。
$ git clone https://github.com/liuxiaoyu-git/federation-dev
$ cd federation-dev/archive
  1. 由於我們後面部署的測試應用容器需要更高運行權限，因此需要執行以下命令爲test-namespace項目提權。
$ oc --context=cluster1 -n test-namespace adm policy add-scc-to-user anyuid -z default
securitycontextconstraints.security.openshift.io/anyuid added to: ["system:serviceaccount:test-namespace:default"]
$ oc --context=cluster2 -n test-namespace adm policy add-scc-to-user anyuid -z default
securitycontextconstraints.security.openshift.io/anyuid added to: ["system:serviceaccount:test-namespace:default"]
  1. 確認“federation-dev/archive/sample-app”中的所有文件內容中的“namespace”設置的是我們本次需要同步的項目“test-namespace”。
  2. 部署聯邦應用資源，包括federatedconfigmap、federateddeployment、federatedsecret、federatedservice、federatedserviceaccount。然後依次查看這些聯邦對象。
$ oc --context=cluster1 -n test-namespace apply -R -f sample-app/
federatedconfigmap.types.kubefed.k8s.io/test-configmap created
federateddeployment.types.kubefed.k8s.io/test-deployment created
federatedsecret.types.kubefed.k8s.io/test-secret created
federatedservice.types.kubefed.k8s.io/test-service created
federatedserviceaccount.types.kubefed.k8s.io/test-serviceaccount created
 
$ oc --context=cluster1 -n test-namespace get federatedconfigmap
NAME             AGE
test-configmap   3m
$ oc --context=cluster1 -n test-namespace get federateddeployment
NAME              AGE
test-deployment   3m
 
$ oc --context=cluster1 -n test-namespace get federatedsecret
NAME          AGE
test-secret   3m
$ o --context=cluster1 -n test-namespacec get federatedserviceaccount
NAME                  AGE
test-serviceaccount   3m
$ oc --context=cluster1 -n test-namespace get federatedservice 
NAME           AGE
test-service   3m
  1. 查看以上每個YAML文件，以federateddeployment.yaml文件爲例，確認對象類型是“FederatedDeployment”；部署的“placement”包括cluster1和cluster2；cluster2的配置通過“overrides將運行Pod副本“/spec/replicas”修改爲“5”份。
$ more sample-app/federateddeployment.yaml
apiVersion: types.kubefed.io/v1beta1
kind: FederatedDeployment
metadata:
  name: test-deployment
  namespace: test-namespace
spec:
  template:
    metadata:
      labels:
        app: nginx
    spec:
      replicas: 3
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - image: nginx
            name: nginx
  placement:
    clusters:
    - name: cluster1
    - name: cluster2
  overrides:
 12. clusterName: cluster2
    clusterOverrides:
    - path: "/spec/replicas"
      value: 5
  1. 查看cluster1和cluster2上的應用運行Pod副本數量。
$ oc --context=cluster1 -n test-namespace get deployment test-deployment
NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
test-deployment   3         3         3            3           11m
$ oc --context=cluster2 -n test-namespace get deployment test-deployment 
NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
test-deployment   5         5         5            5           11m
  1. 由於OpenShift的Route是和每個集羣的域名相關，因此我們先採用命令的方式爲cluster1和cluster2的Service生成各自的Route。
$ oc --context=cluster1 -n test-namespace expose svc test-service
$ oc --context=cluster2 -n test-namespace expose svc test-service
  1. 重複以上（8）-（13）步，將sample-app應用部署到在（7）創建test-namespace1項目中（需要將以上相關步驟中的所有命令和部署YAML文件中的“test-namespace”替換爲“test-namespace1”）。

操控聯邦集羣上的應用部署

控制應用在聯邦集羣中的部署範圍

  1. 執行命令，將cluster2從聯邦集羣中去掉。
$ oc --context=cluster1 -n test-namespace patch federateddeployment test-deployment --type=merge -p '{"spec":{"placement":{"clusters": [{"name":"cluster1"}]}}}'
$ oc get kubefedclusters -n kube-federation-system --context=cluster1
NAME       READY   AGE
cluster1   True    29m
  1. 執行命令，將cluster2加入到聯邦集羣中。
$ oc --context=cluster1 -n test-namespace patch federateddeployment test-deployment --type=merge -p '{"spec":{"placement":{"clusters": [{"name":"cluster1"}, {"name":"cluster2"}]}}}'
$ oc get kubefedclusters -n kube-federation-system --context=cluster1
NAME       READY   AGE
cluster1   True    20m
cluster2   True    1m

控制聯邦集羣中部署的Pod數量

場景1-聯邦部署優先級高於集羣部署

  1. 手動直接將cluster2上應用的Deployment運行的Pod部分改爲3，然後馬上查看Deployment的Pod變化情況。確認“DESIRED”的Pod會先變爲3個，然後又會變爲5，最後“AVAILABLE”也會變爲5。這說明是在聯邦集羣中即便可以通過手動修改特性OpenShift集羣中的應用配置，但是聯邦集羣機制還會根據聯邦集羣上的配置將其修正過來，即FederatedDeployment的優先級高於Deployment。
$ oc --context=cluster2 -n test-namespace patch deployment test-deployment --type=merge -p '{"spec":{"replicas":"3"}}'
$ oc get deployment test-deployment --context=cluster2 -n test-namespace -w
NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
test-deployment   3         3         3            3           27m
test-deployment   5         3         3            3           27m
test-deployment   5         5         5            3           27m
test-deployment   5         5         5            5           28m

場景2-ReplicaSchedulingPreference優先級高於聯邦部署

在聯邦集羣中，我們可以用ReplicaSchedulingPreference對象來制定在不同集羣上分配部署副本數量的策略。當使用了ReplicaSchedulingPreference，系統將忽略FederatedDeloyment對象中在spec.replicas部分指定的部署副本數量。

  1. 創建包含以下內容的rsp.yaml文件。它定義了一個ReplicaSchedulingPreference，讓應用運行的副本爲6個，且cluster1和cluster2上運行的Pod數量比例是1:2。
apiVersion: scheduling.kubefed.io/v1alpha1
kind: ReplicaSchedulingPreference
metadata:
  name: test-deployment
  namespace: test-namespace
spec:
  targetKind: FederatedDeployment
  totalReplicas: 6
  clusters:
    cluster1:
      weight: 1
    cluster2:
      weight: 2
  1. 執行命令，在test-namespace項目中創建ReplicaSchedulingPreference對象。
$ oc --context=cluster apply -f rsp.yaml
  1. 稍後執行命令，查看cluster1和cluster2的Deployment副本數量。可以確認聯邦集羣中部署這個應用的Pod數量已經成爲6個，且在cluster1和cluster2上運行的Pod數量比例是1:2。
$ oc --context=cluster1 -n test-namespace get deployment
NAME              READY   UP-TO-DATE   AVAILABLE   AGE
test-deployment   2/2     2            2           13h
$ oc --context=cluster2 -n test-namespace get deployment
NAME              READY   UP-TO-DATE   AVAILABLE   AGE
test-deployment   4/4     4            4           13h
  1. 查看名爲test-deployment的federateddeployment。可以看到FederatedDeployment的缺省部署數量（即每個集羣的replicas爲3個）被Override了（即cluster1的replicas爲2、cluster2的replicas爲4）。另外從最下面的Events可以看出KubeFed的federateddeployment-controller先後更新了cluster1和cluster2中的Deployment。
$ oc --context=cluster1 -n test-namespace describe federateddeployment test-deployment
Name:         test-deployment
Namespace:    test-namespace
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"types.kubefed.io/v1beta1","kind":"FederatedDeployment","metadata":{"annotations":{},"name":"test-deployment","namespace":"t..."
API Version:  types.kubefed.io/v1beta1
Kind:         FederatedDeployment
Metadata:
  Creation Timestamp:  2020-03-21T02:28:30Z
  Finalizers:
    kubefed.io/sync-controller
  Generation:        3
  Resource Version:  1285158
  Self Link:         /apis/types.kubefed.io/v1beta1/namespaces/test-namespace/federateddeployments/test-deployment
  UID:               a7a800d7-6b1b-11ea-abac-1222d546a5b5
Spec:
  Overrides:
    Cluster Name:  cluster1
    Cluster Overrides:
      Path:        /spec/replicas
      Value:       2
    Cluster Name:  cluster2
    Cluster Overrides:
      Path:   /spec/replicas
      Value:  4
      Path:   /spec/template/spec/containers/0/image
      Value:  nginx:1.17.0-alpine
      Op:     add
      Path:   /metadata/annotations
      Value:
        Foo:  bar
      Op:     remove
      Path:   /metadata/annotations/foo
  Placement:
    Clusters:
      Name:  cluster2
      Name:  cluster1
  Template:
    Metadata:
      Labels:
        App:  nginx
    Spec:
      Replicas:  3
      Selector:
        Match Labels:
          App:  nginx
      Template:
        Metadata:
          Labels:
            App:  nginx
        Spec:
          Containers:
            Image:  nginx
            Name:   nginx
Status:
  Clusters:
    Name:  cluster2
    Name:  cluster1
  Conditions:
    Last Transition Time:  2020-03-21T15:38:30Z
    Last Update Time:      2020-03-21T15:38:30Z
    Status:                True
    Type:                  Propagation
Events:
  Type    Reason           Age                      From                            Message
  ----    ------           ----                     ----                            -------
  Normal  UpdateInCluster  32m (x2 over 46m)        federateddeployment-controller  Updating Deployment "test-namespace/test-deployment" in cluster "cluster1"
  Normal  UpdateInCluster  7m35s (x172 over 3h47m)  federateddeployment-controller  Updating Deployment "test-namespace/test-deployment" in cluster "cluster2"

場景3-關閉聯邦集羣中的一個集羣

在完成以上場景2後，如果關閉聯邦集羣中的一個集羣（例如cluster2），KubeFed會將6個Pod全部運行在cluster1上。

kubefedctl 命令

kubefedctl enable和kubefedctl disable子命令

這兩個命令可以分別打開和關閉在聯邦集羣中傳播Kubernetes API類型。

kubefedctl federate子命令

Kubernetes API Type轉成Federated Type

我們可以用“kubefedctl federate”命令將一個一般的Kubernetes API Type轉成Federated Type，例如將configmap變成federatedconfigmap。在實現過程中有以下兩種方法：

1. 根據YAML定義文件

  1. 查看一個定義ConfigMap對象的YAML文件configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: web-file
data:
  index.html: "Hello from Kubernetes Cluster Federation!"
  1. 用“kubefedctl federate”命令轉換成FederatedConfigMap，輸出便是FederatedConfigMap的YAML。可以根據輸出結果創建FederatedConfigMap。
$ kubefedctl federate -f configmap.yaml
---
apiVersion: types.kubefed.io/v1beta1
kind: FederatedConfigMap
metadata:
  name: web-file
spec:
  placement:
    clusterSelector:
      matchLabels: {}
  template:
    data:
      index.html: Hello from Kubernetes Cluster Federation!

2. 根據已有對象，生成Federated類型的對象

  1. 執行命令，根據當前已有名爲web-file的ConfigMap對象，生成對應的同名FederatedConfigMap對象。
$ kubefedctl federate cm web-file
W0322 01:57:10.549787    3583 federate.go:410] Annotations defined for ConfigMap "test-namespace/web-file" will not appear in the template of the federated resource: map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"v1","data":{"index.html":"Hello from Kubernetes Cluster Federation!"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"web-file","namespace":"test-namespace"}}
]
I0322 01:57:10.560774    3583 federate.go:503] Successfully created FederatedConfigMap "test-namespace/web-file" from ConfigMap
  1. 確認已經生成FederatedConfigMap對象。
$ oc get fcm web-file
NAME       AGE
web-file   2m32s

刪除Federated類型的對象

如果刪除Federated類型的對象，那麼會在聯邦集羣範圍內的所有相關集羣中刪除其對應的Kubernetes API對象。例如當執行命令刪除了fsvc（即FederatedService），OpenShift會刪除cluster1和cluster2中和FederatedService對應的Service對象。

$ oc delete fsvc --all
federatedservice.types.kubefed.io "test-service" deleted

參考文檔

  1. https://github.com/kubernetes-sigs/kubefed/blob/master/docs/userguide.md
  2. https://github.com/openshift/federation-dev/tree/master/archive
  3. https://github.com/openshift/federation-dev/blob/master/archive/docs/ocp4-namespace-scoped.md
  4. https://github.com/openshift/federation-dev/blob/master/archive/docs/ocp4-cluster-scoped.md
  5. https://github.com/redhat-gpte-devopsautomation/rhte2019_multi_cloud
  6. https://www.linkedin.com/pulse/openshift-4x-foundations-fundamentals-kubefed-dewan-i-ahmed
  7. https://juejin.im/post/5dafe985f265da5b560e0d2d
  8. https://juejin.im/post/5db6a43be51d452a01253454
  9. https://www.kubernetes.org.cn/5702.html
發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

Hands-on Lab (11-1) - 使用Prometheus Operator監控應用

文章目錄監控OpenShift集羣監控應用部署被監控的應用通過Operator安裝並配置Prometheus 監控OpenShift集羣 在OpenShift 4 的控制檯中已經在集成了一個Prometheus運行環境，這個內置的

dawnsky.liu 2020-06-22 10:20:25

OpenShift 4.3 - 基於虛擬機的BareMetal離線安裝（Shell說明）

OCP 4.3 Install on KVM as BareMetal - Shell說明

dawnsky.liu 2020-07-07 09:28:52

OpenShift 4.3 - 基於虛擬機的BareMetal離線安裝（7-9）

OCP 4.3 Install on KVM as BareMetal (7-9)

dawnsky.liu 2020-07-07 09:28:52

OpenShift 4.3 - 基於虛擬機的BareMetal離線安裝（1-2）

OCP 4.3 Install (1-2)

dawnsky.liu 2020-07-07 09:28:52

OpenShift 4.3 - 基於虛擬機的BareMetal離線安裝（4）

OCP 4.3 Install on KVM as BareMetal (4)

dawnsky.liu 2020-07-07 09:28:52

OpenShift 4.3 - 基於虛擬機的BareMetal離線安裝（5-6）

OCP 4.3 Install on KVM as BareMetal (5-6)

dawnsky.liu 2020-07-07 09:28:52

OpenShift 4.4 - 用CodeReady開發雲原生應用（1）- 將可執行Quarkus應用部署到OpenShift

文章目錄創建CodeReady項目Live Coding打包Java應用編譯成可執行應用配置服務健康檢查向OpenShift部署Quarkus應用 說明： 本系列參考了https://github.com/RedHat-Midd

dawnsky.liu 2020-06-28 07:09:09

OpenShift 4 - 用容器提升MySQL的可用性

文章目錄在容器中部署MySQL驗證容器化MySQL的可用性 在容器中部署MySQL 創建mysql項目 $ oc new-project mysql 登錄OpenShift控制檯，進入Developer視圖，然後在左側菜單

dawnsky.liu 2020-06-23 00:32:04

OpenShift 4.3 之Knative-Tutorial(7) Eventing之Broker和Trigger

文章目錄概念操作說明部署Broker部署Service部署Event Source部署Trigger測試驗證利用Event Source驗證手動發請求驗證參考 概念 Knative Eventing的Broker和Trigger對

dawnsky.liu 2020-06-22 10:20:25

OpenShift 4.3 之Knative-Tutorial(5) Eventing之Source和Sink

文章目錄環境確認配置Source和Sink參考 環境確認 在OpenShift 4.3中確認Knative的運行環境，包括Knative Serving和Knative Eventing，以及基礎環境Service Mesh環境。

dawnsky.liu 2020-06-22 09:39:21

OpenShift 4.3 之Knative-Tutorial(6) Eventing之Channel和Subscription

文章目錄概念操作參考 概念 Knative Eventing中的Channel和Subscription類似JMS消息體系中的Topic/Subscribe機制。Event Source是消息的發佈者（類似JMS的Publishe

dawnsky.liu 2020-06-22 09:39:21

OpenShift 4.3 之Knative-Tutorial(4) 自動擴展和收縮

文章目錄自動擴展和收縮功能說明配置自動擴展收縮配置擴展收縮上下限 自動擴展和收縮功能說明 在自動擴展和收縮過程中，Revision 有三種狀態： Active：當他們活躍地服務請求時。在Active狀態下，每個Revision都

dawnsky.liu 2020-06-22 09:39:19

OpenShift 4.3 通過腳本配置ServiceMesh和Serverless運行環境

文章目錄Install CatalogSourceConfigInstall ServicemeshInstall Knative ServingInstall Knative Eventing Install CatalogSo

dawnsky.liu 2020-06-22 09:39:19

OpenShift 4.3 之Knative-Tutorial(2) 部署Hello應用並基於Revision/Tag/Traffic實現藍綠部署和恢復發佈

文章目錄部署並測試Hello的Serverless應用藍綠部署和恢復發佈 部署並測試Hello的Serverless應用 創建knative-demo項目。 $ oc new-project knative-demo 部署

dawnsky.liu 2020-06-22 09:39:19

GitOps（9）使用ArgoCD Operator

新建argocd項目。 由於在後面部署的測試應用Pod需要更高的權限，所以需要執行以下命令對當前項目提權。 $ oc adm policy add-scc-to-user anyuid -z default 在OpenSh

dawnsky.liu 2020-06-22 09:39:18