我們爲什麼從 Lambda 遷移到了 ECS？

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本文中，我將深入探討 Lambda 的成功之處，我們面臨的挑戰，以及爲什麼我們最終會決定將一些服務從 Lambda 遷移到 AWS 彈性容器服務（Elastic Container Service，ECS）。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"我們要解決的問題是什麼？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"簡單介紹一下背景，我們的產品是 B2B 軟件公司的一個集成平臺，我們幫助軟件公司構建集成，並將這些集成部署給他們的客戶。一次簡單的集成可能如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"步驟一：在 Dropbox 中下拉 XML 文檔。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"步驟二：使用一些自定義的 JavaScript 代碼處理 XML。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"步驟三：使用一些存儲的憑證，向第三方 API 發佈經過處理的數據。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用戶可以按計劃配置集成以運行，也可以通過 webhook 觸發集成，而平臺則負責運行、記錄和監控集成（以及一大堆其他內容）。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"早期情況"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Prismatic 的第一個實現使用 "},{"type":"link","attrs":{"href":"https:\/\/localstack.cloud\/","title":"xxx","type":null},"content":[{"type":"text","text":"LocalStack"}]},{"type":"text","text":"。我們希望最終把 Prismatic 託管在 AWS 上（根據需要可能會遷移到 Azure、GCP 等），所以需要在本地啓動我們的平臺來模擬 AWS。類似 AWS Lambda 的 LocalStack 服務易於迭代，並且在運行中不會出現大問題。這爲我們提供了一個非常好的開發反饋迴路，我們很快進行了原型設計和測試。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在使用 Lambda 執行集成的每一個步驟時，步驟利用 SQS 傳遞數據並觸發下一個步驟。所以，集成的執行情況如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"執行 Dropbox 的“獲取文件”動作以抓取 XML 文件。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 SQS 中保存 XML 文件的內容，觸發下一步驟。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"運行客戶自定義 JavaScript 代碼以處理 XML。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 SQS 中保存生成的轉換數據，並觸發下一步驟。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"執行一個動作，將處理後的數據發佈到第三方 API。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"保存上一步驟的結果，觸發集成結束。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這對於 LocalStack 來說是一個非常快速的過程。我們可以定義一個 6 步集成，運行它，然後在幾秒內就能看到結果。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"向實際的 AWS Lambda 遷移"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當我們的概念被證明可行之後，我們就將 Prismatic 轉到實際生產環境中，使用實際的 Lambda、隊列、數據庫等等。我們還只是一個小團隊，不想花太多的時間來解決 DevOps-y 的基礎設施問題。我們希望花更多的時間在覈心商品上，而 Lambda 能讓我們做到這一點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此外，Lambda 在很多方面都比較有優勢。比如，我們無需擔心 CPU 或內存分配、服務器監控或自動擴展，因爲這些都是內置的。還可以將包含 JavaScript 代碼的 .zip 文件放到 Lambda 上，AWS 負責剩下的工作。Lambda 讓我們將代碼分成一系列服務（一個用於日誌記錄、一個用於 OAuth 密鑰更新、一個用於集成出錯時短信 \/ 郵件提醒等），這樣我們就能很好地理解由哪些代碼負責哪些任務。成本也很合理：你只需支付計算時間的費用。我們無需全天候運行服務器，只需在我們的原型執行某些任務時付費。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 "},{"type":"link","attrs":{"href":"https:\/\/www.terraform.io\/","title":"xxx","type":null},"content":[{"type":"text","text":"Terraform"}]},{"type":"text","text":" 上運行幾天之後，我們在 AWS 上有了 Prismatic 的第二個實現。集成運行器運行在實際的 Lambda 上，並且被 SQS 觸發。此時，我們就要面對集成運行器的性能問題了。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"爲何 Lambda 無法工作"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 Lambda 中，我們遇到了速度、SQS 大小的限制以及缺少進程隔離等問題。這讓我們重新考慮它作爲集成運行器的有效性。下面我們依次對這些問題進行討論："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"速度。"},{"type":"text","text":"我在前文提到過，在 LocalStack 內運行 6 步集成只需幾秒。但是在實際的 Lambda 和 AWS 中，這花了整整一分鐘。實際上，Lambda 調用非常快，通常爲幾毫秒。但是，在向 SQS 寫入步驟結果的過程中，以及在接下來執行下一步驟的過程中，每一步驟都需要幾秒鐘。對於更爲複雜的集成來說，比如那些循環了 500 多個文件的集成，這就成爲一個障礙：誰想要花上幾分鐘（幾小時）來完成他們的集成？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了加快 Lambda 調用的速度，我們嘗試了許多方法。根據指導原則，我們爲一些 Lambda 實例保留了“熱度”，並將驅動 Lambda 的虛擬 CPU 數量增加到了當時能夠達到的最高水平（6 個虛擬 CPU\/10 GB 內存），但這隻會降低我們集成運行時間的個位數百分比。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"SQS 的大小限制。"},{"type":"text","text":"SQS 限制消息大小爲 256 KB。在集成的各步驟之間傳遞的數據量通常會超過這個大小（畢竟，集成開發人員現在一個數兆的 JSON 文件進行處理是完全合理的）。要想繞過這個大小限制，推薦的解決方案是向 S3 寫入有效載荷，然後通過 SQS 傳遞對 S3 對象的引用——但是這個對 S3 的 API 調用只會使速度更慢。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"結果表明，如果客戶在其集成中編寫了像 global.XMLHttpRequest = null; 這樣不好的代碼，依賴 XMLHttpRequest 庫的集成隨後將在同一個 Lambda 上運行，這將導致錯誤。這個問題很大，因爲一個客戶可能會破壞其他客戶的 "},{"type":"link","attrs":{"href":"https:\/\/www.npmjs.com\/package\/axios","title":"xxx","type":null},"content":[{"type":"text","text":"axios"}]},{"type":"text","text":"。有些客戶甚至可能會惡意執行 global.console.log = (msg) => { nefariousCode(); } 之類的操作，同時，當調用 console.log() 時，同一 Lambda 上執行的其他集成將運行 nefariousCode()。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了解決共享執行空間的問題，我們做了一些嘗試。我們曾強迫 Lambda 每次都冷啓動（這是一個壞主意，原因顯而易見），也曾試過在 "},{"type":"link","attrs":{"href":"https:\/\/www.thegeekdiary.com\/understanding-chroot-jail\/","title":"xxx","type":null},"content":[{"type":"text","text":"chroot jail"}]},{"type":"text","text":" 中啓動不同的 Node 進程。這兩種方法都不起作用：在 Lambda 中啓動子 Node 進程需要 3~5 秒的時間，並且在一定程度上與 Lambda 的初衷背道而馳。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"向 ECS 遷移"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在開發方面，Lambda 爲我們提供了很好的服務：我們可以快速迭代，並獲得一個原型，但是由於 Lambda 面臨的各種問題，我們決定咬緊牙關，花一些開發時間在雲基礎設施上。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了擴展已有的 Terraform 腳本，並將集成運行器遷移到 AWS 彈性容器服務（ECS），我們的團隊已經開始了工作。使用 ECS 容器，我們可以輕鬆地、快速地使用 chroot，並將 Node 進程彼此隔離，從而解決了 Lambda 中出現的進程隔離問題。爲解決 SQS 的大小限制問題，我們改用了 Redis 支持的隊列服務。雖然我們重新發明一些 Lambda 免費提供的“輪子”，比如日誌、自動擴展和健康檢查，但是最終，我們的 6 步測試集成將在 2 秒內恢復運行。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是，ECS 也並不完美，有些東西需要進行取捨。比如，ECS 的自動擴展似乎沒有 Lambda 快。“擴展”似乎要花費一分鐘時間，從 API 調用到 AWS Fargate 獲取並初始化一個準備好接受工作的容器。我們不得不讓一名開發者從產品開發中抽調出來，參與雲基礎設施的工作，同時在 CPU 和內存使用、自動擴展規則和監控方面也有許多問題需要解決，但在產品開發的這一階段，這種痛苦是值得的，因爲它能讓我們爲客戶提供一個快速的集成運行器。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Lambda 還有什麼？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 Lambda 中，我們並沒有移出所有的微服務：許多微服務仍然保留在無服務器的生態系統中，並且在可預見的將來也是如此。集成運行器並不適用於 Lambda，但是對於其他任務，它似乎是一個明確的選擇。在 Lambda 中，我們保留了所有重要的集成服務，而這些服務對 Lambda 的實際執行並不重要。其中包括："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從 ECS 中提取日誌並將其發送到 DataDog 的記錄器服務。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"向 PostgreSQL 數據庫寫入關於集成執行的元數據的服務。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"跟蹤和排隊調度的集成服務。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果用戶的集成出錯，就會向用戶發送短信或電子郵件通知的警報服務。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對第三方服務的 OAuth 2.0 密鑰更新授權服務。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們不希望讓這些服務妨礙集成的運行，如果它們需要更多的一兩秒鐘來運行也沒有問題，因爲對於 Lambda 來說，這樣做是很合適的。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然，隨着時間的流逝，我們的基礎設施肯定會發生變化，但是我認爲我們一直在做正確的決定。通過 LocalStack 的“Lambda”服務，我們可以進行快速地開發和迭代，我們的 AWS 首次部署非常簡單，我們的小型開發團隊可以更改我們的基礎設施，而無需花費大量的開發時間。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Lambda 看起來是一個很有吸引力的解決方案，可以用於託管和擴展我們的微服務，而且對於許多這樣的服務，尤其是那些可能需要一到兩秒才能運行的一步服務，仍然是正確的選擇。但是，對於我們的集成運行器，我們瞭解到 Lambda 的規模、速度和進程隔離限制使得 ECS 成爲更好的選擇，並且值得爲這個特定服務創建 ECS 部署所需的開發時間。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Lambda 讓我們在早期關注產品開發，當時機成熟時，向 ECS 遷移將會非常順利。儘管我們在 Lambda 遇到了很多困難，但我很高興我們走上了正規。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"作者介紹："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Taylor Reece，Prismatic 開發技術推廣工程師，具有 DevOps\/ 雲的背景。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"原文鏈接："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"https:\/\/dzone.com\/articles\/why-we-moved-from-lambda-to-ecs"}]}]}