RocketMQ之Namesrv篇：優雅的服務管理

Namesrv就是RMQ中的路由服務，可以類比ZK在Kafka中的作用，實現了路由管理、服務註冊、服務發現等功能;不過Namesrv相比zookeeper來得要更輕便一點。

一、功能

Namesrv的功能大概可以總結爲下面2點：

接收broker的請求註冊broke路由信息（包括master和slave）：broker啓動時向所有NameSrv註冊，Producer在發送消息前先從Namesrv獲取Broker服務器地址列表，通過負載算法選擇對應服務器進行消息發送。Namesrv與每臺broker保持長連接並進行間隔30s的心跳檢測，broker宕機則從路由註冊表中刪除。
接收client的請求根據某個topic獲取所有到broker的路由信息

二、啓動流程

step1:解析配置文件，填充NameServerConfig,NettyServerConfig屬性值

        final NamesrvConfig namesrvConfig = new NamesrvConfig();
        //這一行是在我本機運行源碼demo所設置的RocketmqHome的值
        namesrvConfig.setRocketmqHome("F:/J.Howie/rocketmq");
        final NettyServerConfig nettyServerConfig = new NettyServerConfig();
        nettyServerConfig.setListenPort(9876);
        if (commandLine.hasOption('c')) {
            String file = commandLine.getOptionValue('c');
            if (file != null) {
                InputStream in = new BufferedInputStream(new FileInputStream(file));
                properties = new Properties();
                properties.load(in);
                MixAll.properties2Object(properties, namesrvConfig);
                MixAll.properties2Object(properties, nettyServerConfig);

                namesrvConfig.setConfigStorePath(file);

                System.out.printf("load config properties file OK, %s%n", file);
                in.close();
            }
        }

        if (commandLine.hasOption('p')) {
            InternalLogger console = InternalLoggerFactory.getLogger(LoggerName.NAMESRV_CONSOLE_NAME);
            MixAll.printObjectProperties(console, namesrvConfig);
            MixAll.printObjectProperties(console, nettyServerConfig);
            System.exit(0);
        }

        MixAll.properties2Object(ServerUtil.commandLine2Properties(commandLine), namesrvConfig);

先創建NameServerConfig(Namesrv業務參數)，NettyServerConfig(Namesrv網絡參數)，然後再啓動時把指定的配置文件或啓動命令中的選項值填充到對象中。

-c configFile 通過-c 命令指定配置文件的路勁
使用 “-- 屬性名屬性值”,例如 --listenPort 9876

Step2：根據啓動屬性創建NamesrvController實例，並初始化該實例(step1中的2官網config都是NamesrvController的屬性)，NamesrvController實例即Namesrv核心控制器

    public boolean initialize() {
        this.kvConfigManager.load();
        //創建NettyServer網絡處理對象
        this.remotingServer = new NettyRemotingServer(this.nettyServerConfig, this.brokerHousekeepingService);
        this.remotingExecutor =
            Executors.newFixedThreadPool(nettyServerConfig.getServerWorkerThreads(), new ThreadFactoryImpl("RemotingExecutorThread_"));
        this.registerProcessor();
        //心跳1：每隔10s掃描一次Broker，移除處於不激活狀態的broker
        this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
            @Override
            public void run() {
                NamesrvController.this.routeInfoManager.scanNotActiveBroker();
            }
        }, 5, 10, TimeUnit.SECONDS);
        //心跳2：每隔10min打印一次kv配置
        this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
            @Override
            public void run() {
                NamesrvController.this.kvConfigManager.printAllPeriodically();
            }
        }, 1, 10, TimeUnit.MINUTES);
        
        return true;
    }

Step3:註冊JVM鉤子函數並啓動服務器，以便監聽broker、消息生產者的網絡請求

    //提示：代碼中如果使用線程池，可以使用JVM鉤子函數（shutdownHook）能夠優雅的關閉線程池，及時釋放資源
    Runtime.getRuntime().addShutdownHook(new ShutdownHookThread(log, new Callable<Void>() {
        @Override
        public Void call() throws Exception {
            controller.shutdown();
            return null;
        }
    }));
    
    controller.start();

三、RouteInfoManager

Namesrv用來存儲路由的基礎信息都放在RouteInfoManager類中,RouteInfoManager類也可以看做是Namesrv的資源類，很多操作都是對此類中的數據進行實時更改：

    private static final InternalLogger log = InternalLoggerFactory.getLogger(LoggerName.NAMESRV_LOGGER_NAME);
    private final static long BROKER_CHANNEL_EXPIRED_TIME = 1000 * 60 * 2;
    private final ReadWriteLock lock = new ReentrantReadWriteLock();
    private final HashMap<String/* topic */, List<QueueData>> topicQueueTable;
    private final HashMap<String/* brokerName */, BrokerData> brokerAddrTable;
    private final HashMap<String/* clusterName */, Set<String/* brokerName */>> clusterAddrTable;
    private final HashMap<String/* brokerAddr */, BrokerLiveInfo> brokerLiveTable;
    private final HashMap<String/* brokerAddr */, List<String>/* Filter Server */> filterServerTable;

topicQueueTable：Topic消息隊列路由信息，消息發送時根據路由表進行負載均衡
brokerAddrTable：Broker基礎信息，包含brokerName、所屬集羣名稱、主備Broker地址
clusterAddrTable：Broke集羣信息，存儲集羣中所有Broker名稱
brokerLiveTable：Broker狀態信息，NameServer每次收到心跳包時會替換該信息
filterServerTable：Broker上的FilterServer列表，用於類模式消息過濾

QueueData、BrokerData、BrokerLiveInfo的類圖：

運行時結構圖：

TopicQueueTable、BrokerAddrTable運行時內存結構：

BrokerLiveTable、ClusterAddrTable運行時內存結構：

四、Namesrv與broker間的心跳

RMQ路由註冊時通過Broker與Namesrv的心跳功能實現的，Broker啓動10s後每間隔30s向集羣中所有的Namesrv發送心跳包，Namesrv會根據收到的心跳包更新brokerLiveTable緩存中BrokerLiveInfo的lastUpdateTimestamp，然後Namesrv每隔10s掃描brokerLiveTable，如果連續120s沒有收到心跳包，則踢除該broker並關閉socket連接。

4.1、Broker發送心跳包：

Broker端心跳包發送（BrokerController#start）

    this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
        @Override
        public void run() {
            try {
                BrokerController.this.registerBrokerAll(true, false, brokerConfig.isForceRegister());
            } catch (Throwable e) {
                log.error("registerBrokerAll Exception", e);
            }
        }
    }, 1000 * 10, Math.max(10000, Math.min(brokerConfig.getRegisterNameServerPeriod(), 60000)), TimeUnit.MILLISECONDS);

4.2、Namesrv處理心跳包

step1：添加寫鎖，防止併發修改路由表，然後判斷該broker集羣是否存在，不存在則創建並添加到broker集合中

    this.lock.writeLock().lockInterruptibly();

    Set<String> brokerNames = this.clusterAddrTable.get(clusterName);
    if (null == brokerNames) {
        brokerNames = new HashSet<String>();
        this.clusterAddrTable.put(clusterName, brokerNames);
    }
    brokerNames.add(brokerName);

step2：維護BrokerData信息,先從brokerAdd人Table獲取broker信息，不存在則新建BrokerData並放入到brokerAddrTable,registerFirst設置爲true;如果存在，直接替換原先的，register設置爲false，表示非第一次註冊。

    BrokerData brokerData = this.brokerAddrTable.get(brokerName);
    if (null == brokerData) {
        registerFirst = true;
        brokerData = new BrokerData(clusterName, brokerName, new HashMap<Long, String>());
        this.brokerAddrTable.put(brokerName, brokerData);
    }
    String oldAddr = brokerData.getBrokerAddrs().put(brokerId, brokerAddr);
    registerFirst = registerFirst || (null == oldAddr);

step3：如果broker爲master,並且brokerTopic配置信息發生變化或者是初次註冊，則需要創建或更新topic路由元數據並填充topicQueueTable。

    if (null != topicConfigWrapper
        && MixAll.MASTER_ID == brokerId) {
        if (this.isBrokerTopicConfigChanged(brokerAddr, topicConfigWrapper.getDataVersion())
            || registerFirst) {
            ConcurrentMap<String, TopicConfig> tcTable =
                topicConfigWrapper.getTopicConfigTable();
            if (tcTable != null) {
                for (Map.Entry<String, TopicConfig> entry : tcTable.entrySet()) {
                    this.createAndUpdateQueueData(brokerName, entry.getValue());
                }
            }
        }
    }

step4：更新BrokerLiveInfo,存活Broker信息表,BrokerLiveInfo是執行路由刪除的重要依據。

    BrokerLiveInfo prevBrokerLiveInfo = this.brokerLiveTable.put(brokerAddr,
        new BrokerLiveInfo(
            System.currentTimeMillis(),
            topicConfigWrapper.getDataVersion(),
            channel,
            haServerAddr));
    if (null == prevBrokerLiveInfo) {
        log.info("new broker registered, {} HAServer: {}", brokerAddr, haServerAddr);
    }

step5：註冊broker的過濾器Server地址列表,一個Broker上會關聯多個FilterServer消息過濾服務器;如果此broker爲從節點，則需要查找該broker的master節點信息，並更新對應的masterAddress屬性。

    if (filterServerList != null) {
        if (filterServerList.isEmpty()) {
            this.filterServerTable.remove(brokerAddr);
        } else {
            this.filterServerTable.put(brokerAddr, filterServerList);
        }
    }

    if (MixAll.MASTER_ID != brokerId) {
        String masterAddr = brokerData.getBrokerAddrs().get(MixAll.MASTER_ID);
        if (masterAddr != null) {
            BrokerLiveInfo brokerLiveInfo = this.brokerLiveTable.get(masterAddr);
            if (brokerLiveInfo != null) {
                result.setHaServerAddr(brokerLiveInfo.getHaServerAddr());
                result.setMasterAddr(masterAddr);
            }
        }
    }

五、其他功能

上面第四章介紹了broker和Namesrv的心跳機制，也可以看做是Namesrv的路由註冊功能。下面介紹下Namesrv的其他功能：

5.1、路由刪除

上文提到，Broker每隔30s向Namesrv發送一個心跳包,Namesrv會每隔10s掃描brokerLiveTable狀態表,如果BrokerLive的lastUpdateTimestamp的時間戳距當前時間超過120s，則認爲broker失效並移除關閉，同時更新其他狀態信息。
路由刪除會從topicQueueTable、brokerAddrTable、brokerLiveTable、filterServerTable中刪除與該Broker相關的信息

    public void onChannelDestroy(String remoteAddr, Channel channel) {
        String brokerAddrFound = null;

        if (brokerAddrFound != null && brokerAddrFound.length() > 0) {
            try {
                try {
                	//1、申請寫鎖，根據brokerAddress從brokerLiveTable、filterServerTable移除
                    this.lock.writeLock().lockInterruptibly();
                    this.brokerLiveTable.remove(brokerAddrFound);
                    this.filterServerTable.remove(brokerAddrFound);
                    
                    //2、維護brokerAddrTable,遍歷brokerAdd人Table，從其的BrokerData中的brokerAddres找到具體的Broker，從BrokerData中移除
                    String brokerNameFound = null;
                    boolean removeBrokerName = false;
                    Iterator<Entry<String, BrokerData>> itBrokerAddrTable =
                        this.brokerAddrTable.entrySet().iterator();
                    while (itBrokerAddrTable.hasNext() && (null == brokerNameFound)) {
                        BrokerData brokerData = itBrokerAddrTable.next().getValue();
                        Iterator<Entry<Long, String>> it = brokerData.getBrokerAddrs().entrySet().iterator();
                        while (it.hasNext()) {
                            Entry<Long, String> entry = it.next();
                            Long brokerId = entry.getKey();
                            String brokerAddr = entry.getValue();
                            if (brokerAddr.equals(brokerAddrFound)) {
                                brokerNameFound = brokerData.getBrokerName();
                                it.remove();
                                log.info("remove brokerAddr[{}, {}] from brokerAddrTable, because channel destroyed",
                                    brokerId, brokerAddr);
                                break;
                            }
                        }
                        if (brokerData.getBrokerAddrs().isEmpty()) {
                            removeBrokerName = true;
                            itBrokerAddrTable.remove();
                            log.info("remove brokerName[{}] from brokerAddrTable, because channel destroyed",
                                brokerData.getBrokerName());
                        }
                    }
                    //3、根據BrokerName,從ClusterAddrTable中找到Broker並從集羣中移除，
                    //如果移除後，集羣中不包含任何Broker,則將該集羣從clusterAddrTable中移除
                    if (brokerNameFound != null && removeBrokerName) {
                        Iterator<Entry<String, Set<String>>> it = this.clusterAddrTable.entrySet().iterator();
                        while (it.hasNext()) {
                            Entry<String, Set<String>> entry = it.next();
                            String clusterName = entry.getKey();
                            Set<String> brokerNames = entry.getValue();
                            boolean removed = brokerNames.remove(brokerNameFound);
                            if (removed) {
                                log.info("remove brokerName[{}], clusterName[{}] from clusterAddrTable, because channel destroyed",
                                    brokerNameFound, clusterName);

                                if (brokerNames.isEmpty()) {
                                    log.info("remove the clusterName[{}] from clusterAddrTable, because channel destroyed and no broker in this cluster",
                                        clusterName);
                                    it.remove();
                                }

                                break;
                            }
                        }
                    }
                    //4、根據brokerName，遍歷所有主題的隊列，
                    if (removeBrokerName) {
                        Iterator<Entry<String, List<QueueData>>> itTopicQueueTable =
                            this.topicQueueTable.entrySet().iterator();
                        while (itTopicQueueTable.hasNext()) {
                            Entry<String, List<QueueData>> entry = itTopicQueueTable.next();
                            String topic = entry.getKey();
                            List<QueueData> queueDataList = entry.getValue();

                            Iterator<QueueData> itQueueData = queueDataList.iterator();
                            while (itQueueData.hasNext()) {
                                QueueData queueData = itQueueData.next();
                                if (queueData.getBrokerName().equals(brokerNameFound)) {
                                    itQueueData.remove();
                                    log.info("remove topic[{} {}], from topicQueueTable, because channel destroyed",
                                        topic, queueData);
                                }
                            }

                            if (queueDataList.isEmpty()) {
                                itTopicQueueTable.remove();
                                log.info("remove topic[{}] all queue, from topicQueueTable, because channel destroyed",
                                    topic);
                            }
                        }
                    }
                } finally {
                	//5、釋放鎖
                    this.lock.writeLock().unlock();
                }
            } catch (Exception e) {
                log.error("onChannelDestroy Exception", e);
            }
        }
    }

5.2、路由發現

RMQ路由發現是非實時的，當Topic發生變化後，Namesrv不知道推送給consumer，而是有consumer拉取最新的路由。

    public RemotingCommand getRouteInfoByTopic(ChannelHandlerContext ctx,
        RemotingCommand request) throws RemotingCommandException {
        final RemotingCommand response = RemotingCommand.createResponseCommand(null);
        final GetRouteInfoRequestHeader requestHeader =
            (GetRouteInfoRequestHeader) request.decodeCommandCustomHeader(GetRouteInfoRequestHeader.class);
        //1、調用RouteInfoManager的方法，填充路由表中的值
        TopicRouteData topicRouteData = this.namesrvController.getRouteInfoManager().pickupTopicRouteData(requestHeader.getTopic());

        //2、找到主題對應的路由信息並且該主題爲順序消息，則從NameServerKVconfig中獲取關於順序消息相關的配置填充路由信息；如果沒找到路由信息則使用TOPIC_NOT_EXISTS
        if (topicRouteData != null) {
            if (this.namesrvController.getNamesrvConfig().isOrderMessageEnable()) {
                String orderTopicConf =
                    this.namesrvController.getKvConfigManager().getKVConfig(NamesrvUtil.NAMESPACE_ORDER_TOPIC_CONFIG,
                        requestHeader.getTopic());
                topicRouteData.setOrderTopicConf(orderTopicConf);
            }

            byte[] content = topicRouteData.encode();
            response.setBody(content);
            response.setCode(ResponseCode.SUCCESS);
            response.setRemark(null);
            return response;
        }

        response.setCode(ResponseCode.TOPIC_NOT_EXIST);
        response.setRemark("No topic route info in name server for the topic: " + requestHeader.getTopic()
            + FAQUrl.suggestTodo(FAQUrl.APPLY_TOPIC_URL));
        return response;
    }

參考資料：
RocketMQ核心技術
RocketMQ實戰與原理解析
博客

RocketMQ之Namesrv篇：優雅的服務管理

一、功能

二、啓動流程

三、RouteInfoManager

四、Namesrv與broker間的心跳

4.1、Broker發送心跳包：

4.2、Namesrv處理心跳包

五、其他功能

5.1、路由刪除

5.2、路由發現

vue項目獲取富文本編輯器wangEditor內容導出爲word（html轉word格式並下載）

dotnet C# 創建 X11 應用時設置窗口背景顏色

Navicat安裝與激活教程

TDengine docker安裝方法

vue3組件通信與props

sapui5

Alpine Linux apk add DNS lookup error

部分JDK版本的發佈時間

工作中用到的腳本合集

合併代碼時Beyond Compare設置

RocketMQ之源碼環境篇：搭建Debug環境

Java併發編程和高併發學習總結（一）-大綱

RocketMQ之Namesrv篇：優雅的服務管理

RocketMQ-消息存儲(一)：寫入過程

RocketMQ-消息存儲(四)、消息消費隊列和索引文件構建機制

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結