現代CI系統太複雜，方向跑偏了

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"與幾年前相比，現在的CI平臺要強大得多。總的來說，這是一件好事。藉助強大的CI平臺，軟件公司和開發人員可以更頻繁地發佈更可靠的軟件，這對軟件用戶或客戶來說是有利的。一些集中式CI平臺（如GitHub Actions、GitLab Pipelines和Bitbucket）帶來了規模效益，互聯網提供了有關如何使用它們的信息。只要搜索一下如何在CI平臺Y上執行X操作，就可以找到一些可以直接複製和粘貼的代碼。畢竟，沒有人願意爲了CI配置問題浪費太多時間，他們只是想快速發佈產品。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"現代的CI系統太複雜了"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CI平臺的進步是以增加複雜性爲代價的，我越來越覺得現代CI系統太複雜了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從根本上講，CI平臺是一種遠程代碼執行服務，執行代碼是爲了構建、測試和發佈軟件。因此，CI平臺通常會提供一系列增值特性，讓你能更輕鬆地發佈軟件。這裏有很多不同的方法和商業模式，其中一種常見的增值特性是使用某種類型的配置文件(通常是YAML)，它本身提供了常見的功能，例如配置版本控制系統的檢出，並指定要運行的命令，而我遇到的問題就是從這裏開始。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現代CI平臺的YAML配置很強大。以下是GitHub Actions工作流YAML的一些"},{"type":"link","attrs":{"href":"https:\/\/docs.github.com\/en\/actions\/reference\/workflow-syntax-for-github-actions","title":"","type":null},"content":[{"type":"text","text":"特性"}]},{"type":"text","text":"："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個嵌入式模板系統，可以將源YAML擴展爲最終可用的YAML文檔，包括一種自定義的"},{"type":"link","attrs":{"href":"https:\/\/docs.github.com\/en\/actions\/reference\/context-and-expression-syntax-for-github-actions","title":"","type":null},"content":[{"type":"text","text":"迷你表達式語言"}]},{"type":"text","text":"。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作業觸發器"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"命名變量"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據條件執行作業"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作業之間的依賴關係"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"定義基於Docker的運行時環境"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"加密祕鑰"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"構成每個作業的步驟以及這些步驟應該執行哪些動作"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果我們稍微擴展一下範圍，把GitHub提供的Actions包含進來，就會有："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"執行Git檢出"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"存儲工作流\/作業使用的工件"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"緩存工作流\/作業使用的工件"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"安裝通用編程語言和環境(如Java、Node.js、Python和Ruby)"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"還有更多"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然還有第三方Actions，而且有很多！"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏有很多特性是必需的，我很難說出哪一個是多餘的。所有這些特性對於足夠強大的CI產品來說似乎都是必需的。如果你的產品不提供其中某些特性，就沒有人會用它。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那麼，我想要抱怨的是什麼呢？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我假定一個CI系統複雜到與構建系統變得難以區分。那麼，你能說服我或你自己：GitHub Actions、GitLab CI和其他CI系統都不是構建系統嗎？那些基礎元素都有了，GitHub Actions工作流由作業組成，作業由步驟組成，就像Makefile由規則組成，規則由命令組成。CI系統和構建系統之間主要的區別在於形式和執行模型(傳統上看，構建系統是在本地，是單機的，而CI系統是在遠程，是分佈式的)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然後，我們反過來想：一個構建系統複雜到與CI系統變得難以區分。前面我說過，CI系統是一種遠程執行代碼的服務。雖然從傳統上看，構建系統是在本地運行(因此不是服務)，但現代的構建系統（如Bazel、Buck、Gradle)完全不一樣。Bazel將遠程執行和遠程緩存作爲內置特性，而這些也是現代CI系統的內置功能！如果我用Bazel建立了一個構建系統，然後定義一個服務器端Git推送鉤子，讓遠程服務器觸發Bazel進行構建、運行測試並將結果發佈到某處，那麼這就變成了一個CI系統嗎？我想是的！雖然很粗糙，但我認爲它就是一個CI系統。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果你有仔細閱讀，就會得出這樣的結論：足夠複雜的CI系統和足夠複雜的構建系統在我看來是一樣的。兩者都提供了一個服務器池，提供了通用的計算\/執行功能和構建\/交付軟件的特性，如任務間工件交換、緩存、依賴關係和用於定義任務的迷你語言。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現代CI系統讓我感到困擾的地方是：我覺得自己是在重新創造一個構建系統，並將構建系統的邏輯碎片化了。CI配置不可避免地會轉化爲一堆複雜的YAML，其中包含各種緩存和依賴關係優化，以便保持較短的執行時間和可靠性——就像構建系統一樣。你會發現，你的構建系統有了CI系統的味道，反之亦然。你最終需要管理兩個複雜的平臺\/系統，而不是一個。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因爲構建系統比CI系統更爲一般化（我認爲一個足夠高級的構建系統可以做的事情是一個足夠複雜的CI系統的超集），這意味着如果構建系統足夠高級，那麼CI系統就是冗餘的。所以，這篇文章的標題可以進一步：CI系統不是太複雜了，而是說它們就不應該存在。CI特性應該作爲構建系統的擴展。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除了冗餘問題，我認爲對系統進行統一對用戶來說更爲友好。將CI系統集成到構建系統中（作爲常規開發工作流的一部分），可以更容易地將CI系統的全部功能暴露給開發人員。請想象一下，你可以在不將變更推到遠程服務器的情況下直接運行CI作業，就像在本地進行構建或測試一樣。這樣可以極大地縮短變更週期。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但請不要誤解我的意思，CI系統的某些功能在構建系統中是找不到的（比如集中式結果報告和用於觸發作業的UI\/API），它們絕對是有必要存在的。當然，遠程計算和作業定義對於構建系統來說是完全冗餘的。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"現代CI產品的方向跑偏了"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果你假設構建系統和CI系統之間很相似，就會發現很多現代CI產品（如GitHub Actions、GitLab CI和其他產品）的方向跑偏了：它們被定義成用來運行CI系統的特定領域平臺。實際上，它們應該退後一步，被定位成構建系統（可能還包括批處理作業，比如數據倉庫\/數據管道中常見的那些）所需的更廣泛的通用計算平臺。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這個層面上，每一個CI產品都是不一樣的。我甚至認爲GitHub Actions是一個CI產品，而不是一個平臺。下面我來解釋一下爲什麼。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在我看來，在一個理想的CI平臺上，我能夠要求執行一組特別的任務。我能夠使用API來定義任務，讓平臺運行它們、上傳工件、報告任務結果以便執行其他依賴任務，等等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GitHub Actions有一個"},{"type":"link","attrs":{"href":"https:\/\/docs.github.com\/en\/rest\/reference\/actions","title":"","type":null},"content":[{"type":"text","text":"API"}]},{"type":"text","text":"，可以用來與服務發生交互，但有一個關鍵的特性無法實現，就是用它來定義特定的工作單元：遠程執行服務。定義特定工作單元的唯一方法是將工作流YAML文件提交到代碼庫中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GitLab Pipelines要好一些。GitLab Pipelines支持父子管道（不同管道之間的依賴關係）、多項目管道（不同項目\/代碼庫之間的依賴關係）和動態子管道（在定義新管道的管道作業中生成YAML文件）等特性。動態子管道是一種重要的特性，它們通常將提交的YAML配置與遠程執行服務分離開來。這裏缺少的是一個無需通過父管道\/ YAML就可以實現該功能的API。如果存在這種API，你就可以在GitLab Pipeline之上構建自己的構建\/CI\/批處理系統，減少GitLab Pipeline的YAML配置文件及其創建者的預期對你帶來的約束。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"像GitHub Actions和GitLab Pipelines這樣的CI產品與其說是平臺，不如說是產品，因爲它們都是基於一個通用的遠程執行服務，將一個自成體系的配置機制（YAML文件）和Web UI（以及相應的API）緊密耦合在一起。對於我來說，要將這些產品視爲平臺，它們需要提供通過API來調度計算的能力，不受內置YAML配置機制的限制。GitLab幾乎已經實現了，目前還不清楚GitHub是否(或是否有興趣)朝這個方向發展。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我想順便提一下Taskcluster，作爲GitHub、GitLab等CI產品的反例。這一小節的內容對整篇文章來說並不是最重要的，可以隨意跳過。但如果你想知道爲工程師打造的CI平臺應該是什麼樣子的，或者你是CI平臺的開發者，想要了解一些值得借鑑的想法，那就讀下去吧。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Mozilla的"},{"type":"link","attrs":{"href":"https:\/\/docs.taskcluster.net\/docs","title":"","type":null},"content":[{"type":"text","text":"Taskcluster"}]},{"type":"text","text":"最初是爲了開發Firefox而構建的通用CI平臺。在2014年和2015年推出之時，它是獨一無二的，它的一些原始功能至今還找不到能夠與之媲美的。或許Mozilla申請了什麼專利，但在開源領域，沒有可以與之匹敵的產品，就連我所知道的那些非開源CI平臺也常常無法提供Taskcluster的所有功能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"據我所知，Taskcluster是目前唯一一個適用於大型項目的開放CI平臺。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Taskcluster讓我很喜歡的一點是它提供了用來定義執行單元的核心原語。"},{"type":"link","attrs":{"href":"https:\/\/docs.taskcluster.net\/docs\/manual\/tasks","title":"","type":null},"content":[{"type":"text","text":"任務"}]},{"type":"text","text":"是Taskcluster的核心執行原語，多個任務被連接在一起形成DAG(這與構建系統的工作方式差不多)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們通過向隊列服務發出API請求來創建任務，這個API請求實際上就是在調度這個工作單元。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"定義好的任務實際上就是帶有元數據的計算單元，這些元數據包括任務依賴關係、任務擁有的權限\/範圍，等等。如果你使用過GitHub Actions、GitLab Pipelines，你就會看到很多你熟悉的基本元素：要執行的命令列表、要在Docker映像中執行的命令、構成工件的文件路徑、重試設置，等等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Taskcluster所提供的特性遠遠超過了GitHub、GitLab等產品。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Taskcluster提供了IAM（身份識別與訪問管理）風格的作用域特性來實現訪問控制。作用域控制你可以執行什麼操作、可以訪問什麼服務、可以使用哪些Runner特性(例如是否可以使用ptrace)、可以訪問哪些祕鑰，等等。例如，Firefox的Taskcluster設置是這樣的：不可信任務是無法訪問Firefox構建任務的簽名密鑰的。Taskcluster是我所知道的唯一一個提供了足夠多保護措施的CI平臺，這些保護措施可以降低CI平臺作爲遠程代碼執行服務的風險。Taskcluster的安全模型讓GitHub Actions、GitLab Pipelines和其他常用的CI服務看起來更像是數據泄露和軟件供應鏈漏洞工廠。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Taskcluster支持使用YAML文件來定義任務，不過它已經提供了一個通用的調度API，所以你不需要這麼做。你可以使用自己的配置或前端來定義任務，不過Taskcluster並不關心這些，因爲它是一個真正的平臺。事實上，Firefox在很大程度上避免使用YAML，而是構建了自己的任務定義功能。Firefox代碼庫裏有很多代碼，在運行時將生成數千個離散的任務，這些任務構成了Firefox的構建和版本DAG，並將其中的一些子圖註冊爲Taskcluster任務。這個功能就是它的一個迷你構建系統，Taskcluster平臺承擔了執行\/評估機制的角色。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Taskcluster的模型和能力遠遠超過了GitHub Actions或GitLab Pipelines，有很多偉大的想法值得借鑑。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Taskcluster是一個非常強大的用戶CI，但現在還沒有可供所有人使用的集中式實例(比如像GitHub或GitLab那樣)，而且學習曲線也相當陡峭。所有這些能力都是以複雜性爲代價的。我不能隨意向普通用戶推薦Taskcluster。但如果其他CI產品無法滿足你的需求，你想建立自己的CI平臺，又負擔得起請幾個人來支持你的CI平臺，那麼Taskcluster就值得考慮。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"未來展望"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在我的理想世界裏，存在着一種遠程代碼執行服務平臺，其目的是爲近實時和批處理\/延遲執行的任務提供服務。它可能是爲軟件開發而量身定製的，因爲這些領域的特定特性將其與其他作爲服務工具的通用計算（如Kubernetes、Lambda等）區分開來。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"DAG的概念被融入到執行模型當中，你可以將執行單元定義成圖來獲得依賴關係。你可以定義獨立的、特別的工作單元，也可以定義一組單元，但不像構建系統那樣，需要在整個執行過程中運行代理來協調任務的執行。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在我的理想世界裏，只需要一個DAG來指定所有的構建、測試和發佈任務。沒有N+1系統或配置需要管理，也沒有額外的平臺需要維護，因爲一切都是統一的。通過合併實現了規模經濟，提高了整體效率。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"平臺由worker池子組成，這些worker運行負責執行任務的代理。有些池子用於近實時\/同步RPC風格的調用，有些池子用於調度\/延遲\/異步任務。你可以定義自己的worker池和worker。高級客戶可能會使用由臨時worker(如EC2 Spot實例)組成的自動伸縮組對容量進行伸縮，以相對低的成本滿足需求。當不再需要這些容量時就終止worker，以此來節約成本(Firefox的Taskcluster實例已經這樣做至少6年了)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於最終用戶來說，一個本地構建包含了驅動或調度用以生成所需構建組件的完整任務圖的子集。一個CI構建\/測試由實現該目標所必需的任務圖的子集組成(它可能是本地構建圖的一個超集)。版本的發佈也一樣。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"至於如何配置前端和定義執行單元，平臺只需要提供一個東西：一個可以用來調度\/執行作業的API。但是，爲了讓這個產品對用戶友好，它也應該提供YAML配置文件（就像現在的CI系統那樣）。這樣，大部分用戶可以繼續使用簡化的YAML界面，而高級用戶可以使用底層的調度\/執行API來開發他們自己的驅動程序。人們爲他們的構建系統開發插件，並集成到這個平臺上。有人會將現有的可擴展構建系統（如Bazel、Buck和Gradle）中的節點轉換爲平臺的計算任務，這樣就可以實現構建系統和CI系統(可能還有數據管道之類的東西)的統一。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後，因爲我們討論的是爲軟件開發量身定製的系統，所以我們需要有健壯的結果\/報告API和界面。如果人們看不到這些奇特的分佈式遠程計算在做什麼，那誰會知道它們究竟給我們帶來了哪些好處？這可能很專業，因爲如何跟蹤結果與特定的領域有關。高級用戶可能想要構建自己的結果跟蹤服務，但平臺至少應該提供一個通用的服務(就像GitHub Actions和GitLab Pipelines那樣)。因爲這是一項巨大的增值特性，如果沒有這項特性，很少人會使用你的產品。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是，我所願景的統一化世界並不會解決上面提到的CI複雜性問題：一個足夠大的構建\/CI系統總是具有內在的複雜性，可能需要專門的人來維護。不過，由於複雜的CI系統幾乎總是附加在複雜的構建系統上，因此通過合併構建系統和CI系統可以縮小複雜性的表面積(比如，你不需要操心構建\/CI互操作性問題)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我願景中的所有組件現在都以某種形式存在着。Bazel、Gradle Enterprise和其他現代構建系統都有用於遠程執行和緩存的RPC。它們甚至是可擴展的，你可以開發自己的插件來改變構建系統的核心功能(當然是在不同的程度上)。Taskcluster和GitLab Pipelines支持任務的DAG調度。一些批處理作業執行框架（如Airflow）看起來非常像是特定領域的特別版Taskcluster。我們缺少的是一個可以將所有這些功能捆綁在一起的單一的產品或服務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我確信，我所願景的不是能否實現的問題，而是我們是否應該實現以及誰來實現的問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這可能就是問題的所在。我不想這麼說，但除了少數公司之外，我真的懷疑這種服務能否在短期內成爲一種廣泛可用的服務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我的願景的價值在於統一離散的系統（構建、CI，也許還有一些臨時的系統，如數據管道，這些系統本身就足夠複雜）。出於業務\/效率方面的原因，你需要將它們統一起來。畢竟，如果它們沒有那麼複雜或低效，你可能就不會關心如何讓它們變得更簡單或更快。從這方面講，我們可能已經過濾掉了90%的市場，因爲他們的系統還不夠複雜。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要實現這個願景，需要採用足夠先進的構建系統，充當統一DAG遠程執行服務的大腦。一些公司和項目將採用先進的構建系統（如Bazel），因爲他們有資源、技術知識和效率激勵機制，但其他很多公司不會這麼做。相對於簡單的構建系統，高級的構建系統所提供的額外好處往往是微不足道的。很多公司將構建和CI支持視爲產品開發成本。如果你能夠在一個並不先進的構建系統上取得成功，並且在沒有過多困難的情況下只花費一小部分成本就取得足夠好的成效，那將成爲很多公司和項目相繼效仿的榜樣。人們並不關心有關構建系統和CI系統的爭論是一個怎樣的結果：他們只想發佈產品。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在我看來，這個想法的總體目標市場太小了。在未來幾年內，沒有任何有技術專長的公司能夠實現和提供這樣的服務。我在這個領域工作了10年，見證了Taskcluster模式的潛力，看到了以前的、現在的和潛在的僱主都在爲此不同程度地掙扎着。我知道這個想法對某些人來說是非常有價值的。儘管這對一些公司來說很重要，但我的直覺是，他們只代表了整個潛在市場的一小部分，對於GitHub或GitLab這樣的現有CI產品來說，這塊蛋糕太小了，目前還不足以引起他們的注意。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我不認爲在這個領域創業是個好主意：因爲獲取客戶太難了。而且，由於很多核心技術已經存在於現有的工具中，在專利知識產權方面並沒有什麼護城河可以阻止那些財力雄厚的模仿者。你最好的退出方式可能是被微軟\/GitHub、GitLab或像亞馬遜\/AWS這樣想在這個領域發展的公司收購。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我認爲，我們最希望的是看到現有的CI平臺能夠實現這個願景，並向全世界發佈，或者作爲開源項目或服務提供出來。GitHub、GitLab和其他代碼託管提供商是理想的候選者，因爲它們的社區有助於推動行業的採用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我不確定是什麼時候，但我打賭GitHub\/微軟會率先採取行動。他們在更廣泛的市場\/產品捆綁方面有更強烈的動機(想想他們已經在Visual Studio或GitHub Workspaces中集成構建和CI系統)。此外，他們面臨着一些巨大的構建系統和CI挑戰(特別是Windows)。很明顯，微軟正在GitHub上做開發，甚至是公開的。微軟工程師將感受到離散的構建和CI系統給他們帶來的痛苦和限制。最終，GitHub上至少需要有一個構建系統遠程執行服務。我希望GitHub(或其他公司)將其作爲一個統一的平臺\/服務\/產品而不是離散的服務來實現，因爲正如我所說的，它們實際上是相同的問題。但提供統一的產品並非阻力最小的途徑，所以誰知道會發生什麼。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"結論"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果我有打響指的魔力，可以讓離散的構建、CI（或許還有批處理系統，如數據管道）系統向前快速發展10年，那麼我會："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":1,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"讓Taskcluster成爲最好的遠程執行服務平臺。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"添加對實時同步執行API（如Bazel的遠程執行API）的支持，作爲對現有批處理\/異步能力的補充。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"定義Starlark方言，這樣就可以像Bazel等構建工具中的原語一樣定義CI\/發佈任務。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"迫使其他構建工具（如Bazel）做出改進，縮短構建時間。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":5,"align":null,"origin":null},"content":[{"type":"text","text":"提供用於平臺交互、查看結果報告的高級Web UI。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":6,"align":null,"origin":null},"content":[{"type":"text","text":"向全世界發佈。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這個夢想會很快成爲現實嗎？可能不會，但夢想還是要有的。或許，一些讀者可能會自己去追逐這個夢想。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"https:\/\/gregoryszorc.com\/blog\/2021\/04\/07\/modern-ci-is-too-complex-and-misdirected\/?fileGuid=gN3ndzW02sgExGVh"}]}]}