Source Map在前端監控中的應用和實踐

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Web前端開發中，對於生產環境的代碼通常會進行壓縮和混淆處理，以減小代碼體積並提高源代碼安全性。然而當生產環境有JS錯誤發生時，壓縮和混淆的代碼也讓定位錯誤變得更困難。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們先來看一個愛奇藝智能前端異常監控平臺（鷹眼Hawkeye）檢測到的錯誤日誌：​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/bd/bd35746d0278e8fd2d81e3671f67811b.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖一 Hawkeye 收集的JS錯誤示例","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以看到，壓縮後的錯誤行號和列號已經丟失，函數名也經過了變換，出錯的真實文件名很難確定。如此，根據壓縮代碼的報錯信息是很難定位錯誤的，但是，線上代碼又必須壓縮上傳處理。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何平衡？SourceMap正是解決這個矛盾的利器。本文將從Source Map的基本概念及其在前端異常監控中的運用、設計及實踐等方面進行介紹。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"一、Source Map介紹","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Source Map是一種存儲了源代碼和編譯後代碼映射關係的信息文件，包含代碼轉換前後的位置信息。現在各種主流的打包工具都支持生成Source Map,例如: Uglify JS、Grunt、Gulp、Webpack等。生成Source Map一般在項目打包的編譯階段進行，打包工具先將源代碼解析出AST（Abstract Syntax Tree），在生成編譯後的代碼時，即圖二generate過程中，利用AST中節點的來源信息等生成Source Map。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/51/51190662571d81aceb8fdde9b0de54b7.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖二 編譯並生成Source Map","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"二、JS錯誤採集和利用Source Map解析錯誤","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"JS錯誤一般可以通過window.addEventListener監聽error事件收集，框架錯誤一般利用框架暴露的鉤子函數收集，比如Vue.config.errorHandler。收集到的錯誤中，從Error.prototype.stack可解析出錯誤堆棧信息。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"獲取到錯誤堆棧信息、Source Map文件內容後則可以通過Mozilla的source-map庫進行錯誤定位。該庫中的SourceMapConsumer 實例表示一個已解析的源映射的相關信息，我們可以通過爲這個實例提供文件位置和文件內容來在生成的源中查詢有關原始文件位置的信息。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前，各大監控平臺均有針對錯誤監控的Source Map的解析功能。例如開源監控平臺Sentry支持Webpack插件和自身的CLI工具上傳map文件後進行解析，付費平臺Fundebug支持通過前端UI、自身CLI及接口上傳map文件後進行解析。但是這些平臺也有一些缺點，比如有的不支持使用者選擇文件進行關聯匹配，有的在使用者選擇map匹配不成功時場景交互體驗不友好。我們受之啓發，開發了基於鷹眼監控系統的Source Map功能，主要有以下優勢：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"支持更簡單的配置來通過Webpack的Plugin進行Source Map文件的上傳，並且上傳過程出現異常不影響項目整體編譯流程，代碼侵入性低。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"支持更多的上傳和選擇Source Map文件的方式，以適用於更多的業務錯誤監控的場景，且在使用過程中有更明確的提示和更友好的交互。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"支持map文件管理功能，通過項目名稱和版本號選擇關聯，新版本可以選擇歷史版本的map文件進行匹配。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面對Source Map在監控系統中的運用、鷹眼Source Map功能設計及實踐詳細闡述。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"三、Source Map在監控系統的運用","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"將Source Map運用於前端異常監控系統，主要需要解決的問題便是使用者如何上傳Source Map以及監控系統如何將Source Map和錯誤信息關聯。考慮到易用性,鷹眼SourceMap功能支持三種使用方式：針對單個JS報錯堆棧從手動上傳Source Map、集成插件自動上傳關聯JS錯誤堆棧信息、針對單個JS報錯堆棧從SourceMap列表中選擇關聯。如圖三所示。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ce/ce818623a36d542b91e1da9c03db45ab.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖三 Hawkeye Source Map三大功能","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"相應的，整體架構如下：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e3/e35da58ff28aa5c9169619b6dee2dc5b.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖四 整體架構","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以下是對各個模塊的介紹：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Source Map自動上傳插件(支持Webpack3+)：使用者在項目打包流程中集成此插件，便可將生成的Source Map自動上傳。此插件能夠在不影響項目編譯流程的前提下，在Webpack輸出 asset 到 output目錄之後，獲取打包生成的map文件並執行上傳邏輯。並且可以通過參數配置可以選擇在Webpack編譯完成執行後對map文件進行刪除，不影響上線流程。鑑於歷史項目使用Webpack老版本較多，鷹眼針對Webpack不同的版本進行了兼容優化。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"手動上傳Source Map模塊：使用者可針對單條JS錯誤，上傳單個Source Map進行匹配，並且進行了同名檢測的提示，在能夠自助上傳map外也能通過提示減少錯誤匹配的情況。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Source Map自助選擇模塊：提供了可以通過Source Map管理中心進行選擇map進行關聯的功能，對於不能每次每個版本都能上傳Source Map文件的項目，能夠進行列表和搜索選擇需要匹配的map文件。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Source Map定位展示模塊：展示JS報錯對應的真實源代碼及位置。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Source Map管理中心：集中管理Source Map文件的添加刪除。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上面提到的三種使用方式是互補的關係，其中集成插件自動上傳關聯JS錯誤堆棧信息的方式是非常建議使用者接入的，其優勢在於可方便地集成到項目的自動化構建發佈流程。接入一次之後，收到JS報錯時，便可查看此報錯在源代碼的位置。實現流程如圖五示:","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"SourceMap自動上傳插件拉取項目打包生成的Source Map文件。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"調用文件上傳中心接口上傳文件。文件上傳中心有獨立的鑑權機制、對接了網絡安全服務。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"獲取SourceMap文件存儲信息後連同項目、版本提交至Source Map管理中心。提交完成後可以刪除構建過程中產生的Source Map文件。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用者在前端異常監控系統查看JS報錯時，監控系統根據JS報錯的項目、版本號從SourceMap管理中心獲取Source Map文件。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"調用Mozillasource-map庫進行解析關聯，對源代碼信息進行可視化展示。","attrs":{}}]}]}],"attrs":{}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/59/59503b6cadd2607df198807f2e2752e4.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖五 Hawkeye Source Map自動上傳插件流程設計","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"四、項目中接入","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在項目中接入Source Map自動上傳插件的方法及使用技巧如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"初始化監控平臺接入代碼時需要攜帶versionNum字段，該字段爲當前項目的版本號，以便於後期錯誤堆棧和項目map文件進行版本管理處理。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在項目打包時生成map文件（需要獨立的map文件）。Webpack對於Source Map的生成配置，建議模式使用hidden-source-map，其能夠生成完整獨立的map文件且不會泄露sourceMappingURL，但缺點是會造成編譯耗時增加。cheap-source-map和cheap-module-source-map因省略一些內容，並不會準確完整的還原錯誤位置。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"打包部署項目時集成map上傳插件, 在devDependencies中安裝插件，在項目中的Webpack的plugin配置中增加該插件配置。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"若接入監控平臺時正確傳入版本號和項目名稱，並且在項目打包時通過map上傳插件上傳正確版本及項目名稱的map文件。鷹眼在發生錯誤上報堆棧信息時會自動將JS名稱和map文件進行匹配，顯示其錯誤正確的源碼地址。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"若無法自動匹配時，也可以從Source Map管理中心選擇該項目和版本對應的map文件進行匹配，或者手動上傳map文件自行匹配。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"在鷹眼監控後臺的使用方式如圖六標註示意：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/5c/5c5172d338b70631e1340fc984991369.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖六 Hawkeye Source Map功能接入示例","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"五、總結與展望","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們再看開篇提到的錯誤，在鷹眼管理系統中Source Map視圖下，可以很方便地看到真實文件名和代碼行號了，如圖七。由於可以直接定位到JS出錯的源代碼位置，接入了此功能的項目，JS錯誤定位排查效率得到了極大的提升。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/1d/1d6faa1f8c535c926adc36a11758c386.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖七 鷹眼JS錯誤SourceMap視圖","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在一些項目生成Source Map以及集成上傳Source Map插件的實踐中，我們也遇到過一些難點：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1、在一些採用自定義Webpack配置打包的項目中，按照官方常規的devtool:hidden-source-map配置總會使得一些打包生成的JS文件體積增大，除了單獨生成的Source Map文件，在一些編譯後的JS中混入了一些SourceMap的內容。我們通過詳細查看這些體積增大的JS文件的特點和內容，最終對vue-loader、css-loader分別指定CSS的Source Map選項關閉解決問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2、微前端項目的Source Map關聯問題。在一些微前端架構項目中，微前端獨立上線，有獨立的版本控制，而採集報錯的Hawkeye JSSDK只需集成在基座項目，微前端打包上線的流程中無法獲取基座的版本，會造成SourceMap文件無法自動關聯匹配。我們採取一種比較簡潔的方式，在採集JSSDK中支持傳入微前端版本的函數，在基座項目中根據微前端的標識等計算獲取微前端的版本，當採集到報錯投遞時，將微前端的版本一同投遞。這樣後臺關聯時，優先匹配微前端版本號，其次匹配基座版本號，即可成功自動關聯。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3、因爲準確的定位錯誤需要上傳完整的獨立map文件，在個別較大的項目中，打包生成Source Map文件時，會影響編譯速度，比較壞的情況下會增加80%的編譯時間。所以後續考慮根據這些項目自身情況，獨立封裝和優化SourceMap生成模塊，而不是通過指定打包工具的選項執行其內置生成流程，儘量減少對項目整體打包耗時的影響。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外，根據接入項目的情況，鷹眼Source Map自動上傳插件後續也會支持更多打包構建工具，比如rollup、gulp。","attrs":{}}]}]}