閒魚前端如何做容災

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"項目背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在我們端側的開發經驗裏，大概都會遇到這些問題："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用戶反饋頁面有時打不開\/白屏；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"服務端接口返回200，但是頁面卻沒有數據；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於服務端依賴的二方庫\/服務出現問題等原因，頁面接口突然抽風出錯，導致頁面展示錯亂或者顯示內容錯誤；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"流量高峯時候服務端扛不住壓力接口掛了；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"服務端監控顯示接口穩定性有99.99%，但是由於網絡、客戶端環境等原因，接口的真正可用性可能要打個折扣; "}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其實，總結起來這些問題都是由於接口可用性差造成的，而提升接口可用性是解決這些問題的不二法門。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"前端如何提升頁面穩定性"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"兜底容災的核心是提前將可用的數據存儲在可靠的地方，在需要時能夠高效便捷地獲取到這份數據。對於這個問題，本地存儲往往會浮現在我們的思路中，但是本地存儲存在可靠性差、存儲容量有限、無法服務端控制等問題。和本地存儲相比，OSS + CDN更符合我們的需求，它穩定、可靠、容量不限、可控性強的特點非常適合用於存儲兜底數據。（備註：阿里雲對象存儲服務，簡稱 OSS，是阿里雲提供的海量、安全、低成本、高可靠的雲存儲服務。）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是當我們在具體實現時，會措手不及地發現很多其他問題："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何及時地更新兜底數據？如何保證兜底數據的正確性？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何在前端組件中請求兜底數據？如何保證請求到的兜底數據是正確的？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何確定OSS資源的地址？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何確定某個接口異常的時候該走兜底？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何區分接口是否適用兜底？如果管理？如何判斷？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何知道某個接口的到底有沒有走兜底容災的流程？兜底容災的成功率如何？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何維護管理遠程的兜底數據"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"等等以上類似問題，但是前端做容災要解決的問題遠遠不只是以上這些。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"前端兜底容災的基本流程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"讓我們先一起來看到前端容災的基本流程， 在設計兜底容災組件時，除了 CDN 容災外，我們還考慮了本地存儲兜底方案以及離線數據兜底方案："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本地存儲兜底方案：使用端的本地存儲能力存儲兜底數據 離線數據方案：在離線包發佈時將接口數據離線爲CDN文件預置其中，作爲兜底數據 如此，兜底容災的基本流程可以簡單描述爲："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/1d\/1d5d0ae1eb065fa4c42d448f621637f9.jpeg","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們將以上流程封裝成前端SDK，然後集成到前端request組件的插件。業務方使用的時候只需要通過參數配置即可引入使用。當然，組件不能僅僅實現功能，還要好用！爲此在組件設計時考慮了很多問題，最後呈現的容災組件具備以下特點："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"支持無縫接入：無論是接口API、參數，還是返回結果，都和目前通用的接口請求組件保持一致。接入只需要新增容災參數即可"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"配置控制：對於兜底配置提供了多個參數，提供了簡單或複雜的配置方式，開發者可以靈活控制兜底容災的流程和規則。對於接口的黑白名單和容災接口管理提供了可視化配置後臺。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"擴展服務：正如流程中所述，組件在關鍵節點採集了日誌信息，藉助這些日誌信息，提供數據分析、自動註冊等服務"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"關於容災地址唯一key計算"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"容災api的地方一定要保證唯一性，主要是參考api地址+參數值+版本號等信息來生成唯一的路由地址，最後將生成的地址轉碼加密；舉個例子：global.alicdn.com\/{HTTPTYPE}\/{API_NAME}\/{API_VERSION}\/{PARAMS}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果是本地容災，如何將過期信息加入到存儲數據裏，我爲了更快捷的處理速度。是將這個信息也加入到本地路由key信息裏的。舉個例子：{UPDATE_TIME}\/{TAG}\/{API_NAME}\/{API_VERSION}\/{PARAMS} 本地數據更新時間+特殊標記符+api名稱+api版本號+相關參數 這樣做的好處就是，當我們需要計算本地當前數據是否過期以及讀取的時候十分方便，只需要遍歷相關key然後和過期時間做對比即可。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這邊關於相關參數的設置，是否一個接口的所有參數都需要用來使用生成唯一的key；那麼顯然也是不合理的，比如pageSize等，一些不會影響到數據準確性的參數。所以這邊還有一個必要參數的概念，只使用必要參數來生成唯一KEY"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"關於數據校驗"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何判斷返回的數據是否“正確”，不論是服務端的數據還是我們的容災數據。基本想到數據校驗，第一反應想到是維護一套數據的schame，好非常完美，每個字段都校驗到了。但是在實際情況下，每個接口都維護和生成一套schame,操作成本和維護成本都是比較高的事情，更糟糕的是，當這個接口數據接口複雜、大的時候，如果對返回的API數據做大而全的數據校驗會影響到頁面性能。結合我們業務的實際情況是，99%以上的場景下，接口異常都是結構化的異常，比如一個接口重要屬性沒有了。這麼一來我們只需要判斷接口的某個重要屬性的正確性就可以了，所以數據校驗這塊最後我們採取是“剋制”的校驗方案去做這個事情。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"關於容災觸發校驗"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"容災觸發校驗主要是校驗兩個方面："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"是否開啓容災"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當前接口異常情況是否適用於容災 過濾掉不能走容災的接口異常類型（比如未登錄、非法請求等接口異常信息） 過濾掉不能走容災的接口（比如個人信息數據，權益信息數據等等 ）"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"相關異常類型的配置我們可以在容災配套的後臺系統進行維護，實時生效。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"關於容災數據配置和維護"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"後臺系統主要提供的是對CDN容災數據的，增刪改查的功能，還有其他相關配置信息維護管理的功能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同時爲了保持容災遠程數據的可用性，還承擔着對這些數據管理以及定時更新的任務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下圖（新增api）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/02\/02139a90a2166926a7a7b1b9ed8d8e79.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"關於數據分析與容災系統監控"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對我們的容災系統運行的到底如何，我們也需要他各個階段進行穩定性監控以及數據分析；主要包括："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"容災成功率"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"容災開啓率"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接口的錯誤率"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接口異常分析"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"系統穩定性分析 "}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當我們發現某個接口的異常率激增的時候，會實時反饋給服務端同學進行異常排除；"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"業務效果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下圖某日接口異常時容災效果："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/7c\/7c5ea4a3e8678527cf6b1bf38ded1f21.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"某天接口整體出現波動，整體錯誤率飆升到 8%以上；經過容災後讓錯誤率降低到0.55%；爲什麼容災不是100%，因爲在統計上我們把接口的網絡異常也計算進來了。CDN容災的方案無法解決，用戶無網絡引起的接口異常問題。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"整體系統架構圖"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/0f\/0f5bd2c6bc1e8c9eaee6499a0a0ce473.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們整體系統架構主要劃分了："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"端側：前端容災SDK"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"容災服務：提供容災數據的管理以及容災服務器的維護能力"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"容災後臺：提供，業務方配置容災和管理數據的可視化平臺"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據埋點監控：對整體接口穩定性以及容災系統穩定性提供數據支持，對異常提供反饋"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"其他"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我作爲開發，一直有一個習慣就是讓我接一個產品或者組件，我接入的成本越少越好。所以在這個過程中，我們也開發了一些瀏覽器插件等幫助大家快速註冊api等方案。當然最好的就是自動接入，關於這塊結合我們自身的業務特點。已經開始去做了。有機會可以再和大家聊一下自動接入的相關流程以及方案和我們遇到的問題。然後以上容災方案最大的問題就是，關於本地容災這塊還是比較單薄，只利用了瀏覽器的存儲能力。關於如何結合客戶端的存儲能力，提升和優化本地容災方案也是即將要做的事情。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文轉載自：閒魚技術（ID：XYtech_Alibaba）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/uL3BfoaZYuFP2nQ_Bd6bDA","title":"xxx","type":null},"content":[{"type":"text","text":"閒魚前端如何做容災"}]}]}]}