OpenKruise：Kubernetes 核心控制器 Plus

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"前言"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在去年的 KubeCon 上海 2019，我有幸在現場見證了 OpenKruise 項目的開源，當時在臺下的我非常興奮，因爲找到了一套讓我的 Kubernetes 集羣的核心資源 Pod 升級和發佈更自動更簡單的方案。如今一年多過去了，前不久 Openkruise 剛發佈了最新的 "},{"type":"codeinline","content":[{"type":"text","text":"v0.6.0"}]},{"type":"text","text":" 版本，目前已經有很多企業在生產環境應用了 OpenKruise，藉助 OpenKruise 提供的自動化能力，大大提升了部署升級效率與質量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"Kruise"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Kruise 是 OpenKruise 中的核心項目之一，Kruise 是 cruise的諧音，字面意義巡航，豪華遊艇（'K' for Kubernetes）。寓意 Kubernetes 上應用的自動巡航，如果把原生 Kubernetes 資源 Deployment 或 StatefulSet 比作小船，那 Kruise 確實就是豪華遊艇了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/00/0013229bbfc263053a5d6c11e854569f.jpeg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Kruise 提供一套在 Kubernetes 核心控制器之外的擴展 workload，但我更願意稱之爲核心控制器 plus。因爲目前 Kruise 提供的一系列 workload，更像是核心控制器資源（Deployment、 StatefulSet、Job 和 DaemonSet）的增強版。比如：Advanced StatefulSet 的介紹裏就寫着是 StatefulSet 的增強版本，在原生 StatefulSet 的基礎上增加了諸多功能。下面筆者就來簡單介紹一下 Kruise 目前提供的所有 workload 控制器，由於篇幅限制每個 workload 的詳細介紹及使用示例將在後續文章中做單獨介紹，本篇只是簡單介紹各個 workload 可能的使用場景及用途。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"CloneSet"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"記得 Kruise 最早還沒有 CloneSet 這個 Workload，所以我之前還是將其歸類爲有狀態應用的控制器增強（最早放出來的是 Advanced StatefulSet），但是自 "},{"type":"codeinline","content":[{"type":"text","text":"v0.4.0"}]},{"type":"text","text":" 版本推出之後，CloneSet 一躍成爲了最受歡迎，使用率最高的 Kruise 控制器之一，同時也補齊了 Kruise 沒有無狀態應用控制器這個短板。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/08/08bc24eb7c86a74df4bd4539f6ea603c.jpeg","alt":null,"title":"","style":[{"key":"width","value":"50%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CloneSet 其實就是 Deployment plus，其提供更加高效、穩定可控的無狀態應用管理和部署能力，支持優雅原地升級、指定刪除、發佈順序、並行/灰度發佈等豐富的策略，可以滿足更多樣化的應用場景。CloneSet 也是目前使用最廣的一類 Kruise 控制器，是 OAM 官方 Kubernetes 套件 crossplane/oam-kubernetes-runtime 支持的 Workload 之一。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CloneSet 的增強分爲兩類："},{"type":"text","marks":[{"type":"strong"}],"text":"擴縮容功能增強"},{"type":"text","text":"和"},{"type":"text","marks":[{"type":"strong"}],"text":"升級功能增強"},{"type":"text","text":"。擴縮容功能增強包括：支持 PVC 模板、指定 Pod 縮容等；升級功能增強包括：原地升級、分批灰度、控制最大不可用數量、控制最大彈性數量、按照不同測控順序升級、發佈暫停等功能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"使用場景"},{"type":"text","text":"：代替原生的 Deployment，將升級過程控制的更加精細、自動和優雅，利用原地升級可以大大降低由於原來重建升級導致的網絡、存儲等方面的損耗，同時還能加快升級速度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Advanced StatefulSet"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Advanced StatefulSet 是 Kruise 最早發佈的控制器之一，是原生 StatefulSet 的增強版本，默認行爲與原生完全一致，在此之外提供了原地升級、並行發佈（最大不可用）、發佈暫停等功能。因爲是在原生基礎上進行開發的，所以只需將原生 StatefulSet 的 "},{"type":"codeinline","content":[{"type":"text","text":"apiVersion"}]},{"type":"text","text":" 由 "},{"type":"codeinline","content":[{"type":"text","text":"apps/v1"}]},{"type":"text","text":" 改爲 "},{"type":"codeinline","content":[{"type":"text","text":"apps.kruise.io/v1alpha1"}]},{"type":"text","text":" 即可完成遷移，非常直接。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Advanced StatefulSet 爲 StatefulSet 提供了和 Deployment 一樣的 "},{"type":"codeinline","content":[{"type":"text","text":"MaxUnavailable"}]},{"type":"text","text":" 策略，可以並行發佈 Pod，而不再像原生 StatefulSet 一樣 one by one 的串行發佈；支持原地升級策略，無需重建 Pod，即可原地升級鏡像，同時也提供了優雅原地升級的策略，控制器在升級前將 Pod status 改爲 not-ready，等待指定時間再升級鏡像，這就爲將 Pod 從 endpoints 端點列表中去除留出了充足的時間（CloneSet 也支持該策略）；還支持指定升級順序、發佈暫停等策略。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"使用場景"},{"type":"text","text":"：代替原生 StatefulSet，有效利用原地升級、並行發佈等功能，提升有狀態應用的發佈速度，爲其配置合適的升級策略，提升發佈速度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"SidecarSet"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/46/462e8b8b896d36d18c475c827c063d9f.jpeg","alt":null,"title":"","style":[{"key":"width","value":"50%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"SidecarSet 的作用就是對 Sidecar 容器做統一管理，支持在一個單獨的 CR 中定義 Sidecar 容器，向將滿足條件的 Pod 中注入指定的 Sidecar 容器，同時 SidecarSet 也支持 Sidecar 容器原地升級。這樣就可以將業務容器和 Sidecar 容器的管理分離，更有利於分工合作，不同的團隊只需關心和自己業務有關的容器，免去了大量的溝通成本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"使用場景"},{"type":"text","text":"：將所有 Sidecar 進行統一管理，一個 CR 管理一類 Sidecar，真正做到業務容器和 Sidecar 容器管理分離，權責清晰。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"UnitedDeployment"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/8e/8e94fb0b740e2edcb2cbdfae46800bae.jpeg","alt":null,"title":"","style":[{"key":"width","value":"50%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"UnitedDeployment 爲由多個區域組成的集羣中實現高可用部署提供了一種新模式，在一個 Kubernetes 集羣中可能存在不同的 node 類型，比如多個可用區、或不同的節點技術（比如 Virtual kueblet）等，這些不同類型的 node 上都有 label/taint 標識。UnitedDeployment 控制器可以提供一個模板來定義應用，每個 UnitedDeployment 下每個區域的 workload 被稱爲 "},{"type":"codeinline","content":[{"type":"text","text":"subset"}]},{"type":"text","text":"，有一個期望的 "},{"type":"codeinline","content":[{"type":"text","text":"replicas"}]},{"type":"text","text":" Pod 數量。目前 subset 支持使用 "},{"type":"codeinline","content":[{"type":"text","text":"StatefulSet"}]},{"type":"text","text":" 和 "},{"type":"codeinline","content":[{"type":"text","text":"Advanced StatefulSet"}]},{"type":"text","text":"。通過 UnitedDeployment 可以同時管理位於多個可用區的同一應用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"使用場景"},{"type":"text","text":"：用於管理跨可用區的有狀態應用，做到管理更精細，過程更自動。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"BroadcastJob"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"BroadcastJob 更像 Job 和 DaemonSet 的組合，BroadcastJob 控制器將 Pod 分發到集羣中每個 node 上，類似於 DaemonSet， 但是 BroadcastJob 管理的 Pod 並不是長期運行的 daemon 服務，而是類似於 Job 的任務類型 Pod。最終在每個 node 上的 Pod 都執行完成退出後，BroadcastJob 和這些 Pod 並不會佔用集羣資源。 這個控制器非常有利於做升級基礎軟件、巡檢等過一段時間需要在整個集羣中跑一次的工作。此外，BroadcastJob 還可以維持每個 node 跑成功一個 Pod 任務。如果採取這種模式，當後續集羣中新增 node 時 BroadcastJob 也會分發 Pod 任務上去執行。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"使用場景"},{"type":"text","text":"：用於管理升級基礎軟件、巡檢等需要在集羣中所有節點或指定類型節點執行的單次任務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Advanced DaemonSet"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Advanced DaemonSet 是 "},{"type":"codeinline","content":[{"type":"text","text":"v0.6.0"}]},{"type":"text","text":" 新增的控制器，是原生 DaemonSet 的增強版本，默認行爲與原生一致，在此之外提供了灰度分批、按 Node label 選擇、暫停、熱升級等發佈策略。遷移方式同 StatefulSet 類似，將 DaemonSet 的 "},{"type":"codeinline","content":[{"type":"text","text":"apiVersion"}]},{"type":"text","text":" 由 "},{"type":"codeinline","content":[{"type":"text","text":"apps/v1"}]},{"type":"text","text":" 改爲 "},{"type":"codeinline","content":[{"type":"text","text":"apps.kruise.io/v1alpha1"}]},{"type":"text","text":" 即可完成遷移。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Advanced DaemonSet 也有 "},{"type":"codeinline","content":[{"type":"text","text":"RollingUpdateDaemonSet"}]},{"type":"text","text":" 的增強策略，同時也提供了多種升級方式如：按照 Selector 標籤選擇升級、分批灰度升級、熱升級和暫停升級。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"使用場景"},{"type":"text","text":"：代替原生 DaemonSet，利用原地升級、灰度升級、選擇性升級以及熱升級等增強特性，更好的維護和管理 DaemonSet 資源。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"結語"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Kruise 項目源自於阿里巴巴多年的大規模應用部署、發佈與管理的最佳實踐，以 "},{"type":"text","marks":[{"type":"strong"}],"text":"automate everything on Kubernetes"},{"type":"text","text":" 爲目標。經過了一年多的發展，OpenKruise 的大部分功能都經過了各種生產環境的洗禮，應用在越來越多的 Kubernetes 系統中，代替原生核心控制器，使應用的升級和管理更加的方便高效。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"參考"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://openkruise.io","title":""},"content":[{"type":"text","text":"OpenKruise - openkruise.io"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a4/a45c3a602d914f821ca970e51f9dcd2d.gif","alt":null,"title":"","style":[{"key":"width","value":"50%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}