TDengine在浙商銀行微服務監控中的實踐

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者：樓永紅 王軒宇 ｜ 浙商銀行","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"浙商銀行股份有限公司（簡稱“浙商銀行\"）是 12 家全國性股份制商業銀行之一，總部設在浙江杭州，全國第13家\"A+H\"上市銀行，致力於打造平臺化服務銀行，爲客戶提供開放、高效、靈活、共享、極致的綜合金融服務。在英國《銀行家》(The Banker) 雜誌“2020 年全球銀行 1000 強\"榜單中，位列第 97 位。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"浙商銀行很早就開始全面推進數字化轉型，2010 年浙商銀行上線電子銀行服務、手機銀行業務，2017 年上線首個區塊鏈服務平臺，同年發佈了直銷銀行品牌。2018 年浙商銀行國標 A 級數據中心啓用，並在 2020 年成立易企銀金融科技子公司。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"業務背景","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"浙商銀行微服務可視化治理平臺是基於Java體系自研的微服務治理監控平臺，爲行內基於統一的微服務框架開發的應用提供全面、實時的微服務治理監控功能，能夠有效提升雲環境下的微服務可視化程度和服務管理控制的便捷度，減少服務故障發現時間，提升故障定位效率，輔助應用程序優化。在這樣的業務場景中，數據量大、監控指標繁雜成了我們的主要挑戰。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過分析監控數據，我們發現它有以下特點：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據高寫入、低查詢、不修改","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於按時間順序插入的監控數據，會涉及到少量的查詢，沒有修改數據的業務場景。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"zerowidth","attrs":{}},{"type":"text","text":"無需事務支持","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果數據模型設計妥當，無需使用事務處理。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"各監控指標之間相互獨立，無需聯合查詢","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據時效性較強，超過一定時間的數據參考價值很小","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"zerowidth","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"核心訴求","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對這樣的場景，我們需要一款能高效處理時序數據的工具。在各種因素的影響下，我們有如下幾點需求：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"開源可控，且最好是國產軟件，且能支持國產化芯片","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"功能穩定","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"社區活躍：開發者社區人員多，問答、討論頻繁且有穩定的開發團隊支持","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"部署成本低：儘可能少的服務節點，運維成本低，佔用 CPU、內存資源較少","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"性能強大：支持萬條數據秒級插入","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"zerowidth","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"時序數據庫選型","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在明確了核心訴求之後，我們調研了幾款典型的時序數據庫產品，包括 ","attrs":{}},{"type":"link","attrs":{"href":"https://druid.apache.org/","title":null,"type":null},"content":[{"type":"text","text":"Apache Druid","attrs":{}}]},{"type":"text","text":"、","attrs":{}},{"type":"link","attrs":{"href":"https://www.influxdata.com/","title":null,"type":null},"content":[{"type":"text","text":"InfluxDB","attrs":{}}]},{"type":"text","text":" 和 ","attrs":{}},{"type":"link","attrs":{"href":"https://www.taosdata.com/cn/","title":null,"type":null},"content":[{"type":"text","text":"TDengine","attrs":{}}]},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"具體對比如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Apache Druid：Druid 是 Apache 基金會旗下的一款高性能的實時分析數據庫，支持時間序列數據。功能強大、可自愈、自平衡、易操作、可進行有效的預聚合和預計算。不過對於時間跨度較大的查詢不夠靈活，而且架構較爲複雜，需要的計算資源多。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"InfluxDB：InfluxDB 是由 InfluxData 公司開發的一款開源的時序數據庫，在業界非常流行。功能強大，部署簡單，使用方便。不過集羣功能沒有開源。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TDengine：能滿足我們的核心訴求，相關測試表明其性能優於 InfluxDB，而且","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/taosdata/TDengine","title":null,"type":null},"content":[{"type":"text","text":"開源","attrs":{}}]},{"type":"text","text":"了核心的集羣功能。簡單快捷、性能強大；支持 JDBC 接口，可以使我們應用快速接入使用。還有一點非常重要，因爲 TDengine 的核心研發團隊在國內，很方便直接交流。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"整體對比之後，我們決定嘗試 TDengine。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"技術架構","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/96/96ffa00205d6348e1a4576fdf065cba5.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可視化服務治理平臺的一個核心功能就是實時的監控功能，監控數據通過微服務框架自帶的埋點發送到數據接收模塊，經過數據處理和格式轉換之後發送到 Apache Kafka。數據處理模塊會接收 Kafka 中的各類監控數據，根據不同的數據類型進行加工計算，存儲到 Elasticsearch 和 TDengine 中。用戶通過前端與後端的服務層對監控數據進行查詢分析。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前爲止，存儲於 TDengine 中的數據主要爲時序類數據，如CPU、內存使用率等系統運行數據，微服務調用、分佈式鎖、數據庫操作處理時間，業務線程池、連接池等各類指標數據。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"數據模型","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前收集的監控數據類型如下：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"JVMGC 相關指標","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"JVMThread 相關指標","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"JVMMemory 相關指標","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"系統負載相關指標","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據庫連接池監控數據","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Transaction 調用監控數據 ","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dubbo 線程池監控數據","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們針對每一個監控類型都採用了超級表建表，以 JVMGC 爲例，建表語句爲：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":" create stable JVMGC (happentime timestamp, jvmUsedMemory bigint, jvmUsedMemoryyPercentage double, nonHeapMemoryUsed double, oldGenUsed bigint, oldGenUsedPercentage double , directBufferUsed bigint) tags (application NCHAR(300), host NCHAR(500), ip NCHAR(50)) ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"與此同時，對每一個監控節點都建立一張子表，這樣處理的好處在於：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 相同指標在進行多表查詢時，可以利用超級表在一個 SQL 語句中完成查詢","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 超級表可以實現表與表之間的數據聚合","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 對於單節點的監控數據，查詢只需要訪問單一子表即可","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"使用體驗","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前微服務可視化服務治理平臺對併發要求較高，但是 TDengine 可以很好地滿足需求，插入/查詢平均耗時均在 10ms 以內。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TDengine 的開源版本小而精緻，開發測試人員可以自己搭建私有的 TDengine 集羣進行測試，無須擔心影響他人，非常方便。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TDengine 的社區非常活躍，遇到問題在 GitHub 上提 issue，或直接在官方的微信交流羣裏討論，都可以快速得到響應。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TDengine 很好地滿足了我們的需求，性能和穩定性都非常出色，同時極大降低了部署和運維成本。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TDengine 的支持團隊也非常積極熱心，能夠快速解決我們遇到的大部分問題，爲我們免除了後顧之憂。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作爲一款爲物聯網場景設計的時序數據庫，TDengine 以存儲效率高、性能強大、功能穩定等特點在傳統互聯網應用架構中同樣有着相當出色的發揮，超級表的設計省去了不少聯表查詢邏輯，大大簡化了業務層的開發工作。接下來，我們也期待在行內挖掘出更多 TDengine 的應用場景。","attrs":{}}]}]}