RocketMQ中PullConsumer的消息拉取源碼分析

在PullConsumer中，有關消息的拉取RocketMQ提供了很多API，但總的來說分爲兩種，同步消息拉取和異步消息拉取

同步消息拉取
以同步方式拉取消息都是通過DefaultMQPullConsumerImpl的pullSyncImpl方法：

private PullResult pullSyncImpl(MessageQueue mq, SubscriptionData subscriptionData, long offset, int maxNums, boolean block,
    long timeout)
    throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
    this.makeSureStateOK();

    if (null == mq) {
        throw new MQClientException("mq is null", null);
    }

    if (offset < 0) {
        throw new MQClientException("offset < 0", null);
    }

    if (maxNums <= 0) {
        throw new MQClientException("maxNums <= 0", null);
    }

    this.subscriptionAutomatically(mq.getTopic());

    int sysFlag = PullSysFlag.buildSysFlag(false, block, true, false);

    long timeoutMillis = block ? this.defaultMQPullConsumer.getConsumerTimeoutMillisWhenSuspend() : timeout;

    boolean isTagType = ExpressionType.isTagType(subscriptionData.getExpressionType());
    PullResult pullResult = this.pullAPIWrapper.pullKernelImpl(
        mq,
        subscriptionData.getSubString(),
        subscriptionData.getExpressionType(),
        isTagType ? 0L : subscriptionData.getSubVersion(),
        offset,
        maxNums,
        sysFlag,
        0,
        this.defaultMQPullConsumer.getBrokerSuspendMaxTimeMillis(),
        timeoutMillis,
        CommunicationMode.SYNC,
        null
    );
    this.pullAPIWrapper.processPullResult(mq, pullResult, subscriptionData);
    if (!this.consumeMessageHookList.isEmpty()) {
        ConsumeMessageContext consumeMessageContext = null;
        consumeMessageContext = new ConsumeMessageContext();
        consumeMessageContext.setConsumerGroup(this.groupName());
        consumeMessageContext.setMq(mq);
        consumeMessageContext.setMsgList(pullResult.getMsgFoundList());
        consumeMessageContext.setSuccess(false);
        this.executeHookBefore(consumeMessageContext);
        consumeMessageContext.setStatus(ConsumeConcurrentlyStatus.CONSUME_SUCCESS.toString());
        consumeMessageContext.setSuccess(true);
        this.executeHookAfter(consumeMessageContext);
    }
    return pullResult;
}

首先通過subscriptionAutomatically方法檢查Topic是否訂閱

public void subscriptionAutomatically(final String topic) {
    if (!this.rebalanceImpl.getSubscriptionInner().containsKey(topic)) {
        try {
            SubscriptionData subscriptionData = FilterAPI.buildSubscriptionData(this.defaultMQPullConsumer.getConsumerGroup(),
                topic, SubscriptionData.SUB_ALL);
            this.rebalanceImpl.subscriptionInner.putIfAbsent(topic, subscriptionData);
        } catch (Exception ignore) {
        }
    }
}

若是沒有就新建一條訂閱數據保存在rebalanceImpl的subscriptionInner中

之後調用pullKernelImpl方法：

public PullResult pullKernelImpl(
    final MessageQueue mq,
    final String subExpression,
    final String expressionType,
    final long subVersion,
    final long offset,
    final int maxNums,
    final int sysFlag,
    final long commitOffset,
    final long brokerSuspendMaxTimeMillis,
    final long timeoutMillis,
    final CommunicationMode communicationMode,
    final PullCallback pullCallback
) throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
    FindBrokerResult findBrokerResult =
        this.mQClientFactory.findBrokerAddressInSubscribe(mq.getBrokerName(),
            this.recalculatePullFromWhichNode(mq), false);
    if (null == findBrokerResult) {
        this.mQClientFactory.updateTopicRouteInfoFromNameServer(mq.getTopic());
        findBrokerResult =
            this.mQClientFactory.findBrokerAddressInSubscribe(mq.getBrokerName(),
                this.recalculatePullFromWhichNode(mq), false);
    }

    if (findBrokerResult != null) {
        {
            // check version
            if (!ExpressionType.isTagType(expressionType)
                && findBrokerResult.getBrokerVersion() < MQVersion.Version.V4_1_0_SNAPSHOT.ordinal()) {
                throw new MQClientException("The broker[" + mq.getBrokerName() + ", "
                    + findBrokerResult.getBrokerVersion() + "] does not upgrade to support for filter message by " + expressionType, null);
            }
        }
        int sysFlagInner = sysFlag;

        if (findBrokerResult.isSlave()) {
            sysFlagInner = PullSysFlag.clearCommitOffsetFlag(sysFlagInner);
        }

        PullMessageRequestHeader requestHeader = new PullMessageRequestHeader();
        requestHeader.setConsumerGroup(this.consumerGroup);
        requestHeader.setTopic(mq.getTopic());
        requestHeader.setQueueId(mq.getQueueId());
        requestHeader.setQueueOffset(offset);
        requestHeader.setMaxMsgNums(maxNums);
        requestHeader.setSysFlag(sysFlagInner);
        requestHeader.setCommitOffset(commitOffset);
        requestHeader.setSuspendTimeoutMillis(brokerSuspendMaxTimeMillis);
        requestHeader.setSubscription(subExpression);
        requestHeader.setSubVersion(subVersion);
        requestHeader.setExpressionType(expressionType);

        String brokerAddr = findBrokerResult.getBrokerAddr();
        if (PullSysFlag.hasClassFilterFlag(sysFlagInner)) {
            brokerAddr = computPullFromWhichFilterServer(mq.getTopic(), brokerAddr);
        }

        PullResult pullResult = this.mQClientFactory.getMQClientAPIImpl().pullMessage(
            brokerAddr,
            requestHeader,
            timeoutMillis,
            communicationMode,
            pullCallback);

        return pullResult;
    }

    throw new MQClientException("The broker[" + mq.getBrokerName() + "] not exist", null);
}

首先通過findBrokerAddressInSubscribe方法查找關於消息隊列的Broker信息

這裏的recalculatePullFromWhichNode方法：

public long recalculatePullFromWhichNode(final MessageQueue mq) {
    if (this.isConnectBrokerByUser()) {
        return this.defaultBrokerId;
    }

    AtomicLong suggest = this.pullFromWhichNodeTable.get(mq);
    if (suggest != null) {
        return suggest.get();
    }

    return MixAll.MASTER_ID;
}

根據消息隊列，在pullFromWhichNodeTable查找其對應的Broker的ID
pullFromWhichNodeTable記錄了消息對了和BrokerID的映射

private ConcurrentMap<MessageQueue, AtomicLong/* brokerId */> pullFromWhichNodeTable =
        new ConcurrentHashMap<MessageQueue, AtomicLong>(32);

（master的BrokerID爲0，slave的BrokerID大於0）

findBrokerAddressInSubscribe方法：

public FindBrokerResult findBrokerAddressInSubscribe(
    final String brokerName,
    final long brokerId,
    final boolean onlyThisBroker
) {
    String brokerAddr = null;
    boolean slave = false;
    boolean found = false;

    HashMap<Long/* brokerId */, String/* address */> map = this.brokerAddrTable.get(brokerName);
    if (map != null && !map.isEmpty()) {
        brokerAddr = map.get(brokerId);
        slave = brokerId != MixAll.MASTER_ID;
        found = brokerAddr != null;

        if (!found && !onlyThisBroker) {
            Entry<Long, String> entry = map.entrySet().iterator().next();
            brokerAddr = entry.getValue();
            slave = entry.getKey() != MixAll.MASTER_ID;
            found = true;
        }
    }

    if (found) {
        return new FindBrokerResult(brokerAddr, slave, findBrokerVersion(brokerName, brokerAddr));
    }

    return null;
}

這裏就根據brokerAddrTable表查找該BrokerID對應的Broker的地址信息，以及是否是slave
封裝爲FindBrokerResult返回

若是沒有找到Broker的路由信息，則通過updateTopicRouteInfoFromNameServer方法向NameServer請求更新，更新完成後再調用findBrokerAddressInSubscribe方法查找

之後會根據相應的信息封裝請求消息頭PullMessageRequestHeader

然後調用pullMessage方法：

public PullResult pullMessage(
    final String addr,
    final PullMessageRequestHeader requestHeader,
    final long timeoutMillis,
    final CommunicationMode communicationMode,
    final PullCallback pullCallback
) throws RemotingException, MQBrokerException, InterruptedException {
    RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.PULL_MESSAGE, requestHeader);

    switch (communicationMode) {
        case ONEWAY:
            assert false;
            return null;
        case ASYNC:
            this.pullMessageAsync(addr, request, timeoutMillis, pullCallback);
            return null;
        case SYNC:
            return this.pullMessageSync(addr, request, timeoutMillis);
        default:
            assert false;
            break;
    }

    return null;
}

這裏就可以看出我前面說的兩種類型，同步拉取和異步拉取

pullMessageSync方法：

private PullResult pullMessageSync(
    final String addr,
    final RemotingCommand request,
    final long timeoutMillis
) throws RemotingException, InterruptedException, MQBrokerException {
    RemotingCommand response = this.remotingClient.invokeSync(addr, request, timeoutMillis);
    assert response != null;
    return this.processPullResponse(response);
}

這裏其實就是通過invokeSync方法，由Netty進行同步發送，將請求發送給Broker
關於消息的發送詳見：
【RocketMQ中Producer消息的發送源碼分析】

在收到響應後由processPullResponse方法處理
processPullResponse方法：

private PullResult processPullResponse(
    final RemotingCommand response) throws MQBrokerException, RemotingCommandException {
    PullStatus pullStatus = PullStatus.NO_NEW_MSG;
    switch (response.getCode()) {
        case ResponseCode.SUCCESS:
            pullStatus = PullStatus.FOUND;
            break;
        case ResponseCode.PULL_NOT_FOUND:
            pullStatus = PullStatus.NO_NEW_MSG;
            break;
        case ResponseCode.PULL_RETRY_IMMEDIATELY:
            pullStatus = PullStatus.NO_MATCHED_MSG;
            break;
        case ResponseCode.PULL_OFFSET_MOVED:
            pullStatus = PullStatus.OFFSET_ILLEGAL;
            break;

        default:
            throw new MQBrokerException(response.getCode(), response.getRemark());
    }

    PullMessageResponseHeader responseHeader =
        (PullMessageResponseHeader) response.decodeCommandCustomHeader(PullMessageResponseHeader.class);

    return new PullResultExt(pullStatus, responseHeader.getNextBeginOffset(), responseHeader.getMinOffset(),
        responseHeader.getMaxOffset(), null, responseHeader.getSuggestWhichBrokerId(), response.getBody());
}

根據響應的狀態，設置PullStatus狀態

然後通過decodeCommandCustomHeader方法，將響應中的信息解碼
最後由PullResultExt封裝消息信息

public class PullResultExt extends PullResult {
    private final long suggestWhichBrokerId;
    private byte[] messageBinary;

    public PullResultExt(PullStatus pullStatus, long nextBeginOffset, long minOffset, long maxOffset,
        List<MessageExt> msgFoundList, final long suggestWhichBrokerId, final byte[] messageBinary) {
        super(pullStatus, nextBeginOffset, minOffset, maxOffset, msgFoundList);
        this.suggestWhichBrokerId = suggestWhichBrokerId;
        this.messageBinary = messageBinary;
    }
    ......
}

public class PullResult {
	private final PullStatus pullStatus;
	private final long nextBeginOffset;
	private final long minOffset;
	private final long maxOffset;
	private List<MessageExt> msgFoundList;
	
	public PullResult(PullStatus pullStatus, long nextBeginOffset, long minOffset, long maxOffset,
	    List<MessageExt> msgFoundList) {
	    super();
	    this.pullStatus = pullStatus;
	    this.nextBeginOffset = nextBeginOffset;
	    this.minOffset = minOffset;
	    this.maxOffset = maxOffset;
	    this.msgFoundList = msgFoundList;
	}
	......
}

拉取到的消息可能是多條，具體內容在PullResult 中的msgFoundList保存，MessageExt是Message的超類

回到pullSyncImpl方法，在拉取到消息後，調用processPullResult方法：

public PullResult processPullResult(final MessageQueue mq, final PullResult pullResult,
    final SubscriptionData subscriptionData) {
    PullResultExt pullResultExt = (PullResultExt) pullResult;

    this.updatePullFromWhichNode(mq, pullResultExt.getSuggestWhichBrokerId());
    if (PullStatus.FOUND == pullResult.getPullStatus()) {
        ByteBuffer byteBuffer = ByteBuffer.wrap(pullResultExt.getMessageBinary());
        List<MessageExt> msgList = MessageDecoder.decodes(byteBuffer);

        List<MessageExt> msgListFilterAgain = msgList;
        if (!subscriptionData.getTagsSet().isEmpty() && !subscriptionData.isClassFilterMode()) {
            msgListFilterAgain = new ArrayList<MessageExt>(msgList.size());
            for (MessageExt msg : msgList) {
                if (msg.getTags() != null) {
                    if (subscriptionData.getTagsSet().contains(msg.getTags())) {
                        msgListFilterAgain.add(msg);
                    }
                }
            }
        }

        if (this.hasHook()) {
            FilterMessageContext filterMessageContext = new FilterMessageContext();
            filterMessageContext.setUnitMode(unitMode);
            filterMessageContext.setMsgList(msgListFilterAgain);
            this.executeHook(filterMessageContext);
        }

        for (MessageExt msg : msgListFilterAgain) {
            String traFlag = msg.getProperty(MessageConst.PROPERTY_TRANSACTION_PREPARED);
            if (traFlag != null && Boolean.parseBoolean(traFlag)) {
                msg.setTransactionId(msg.getProperty(MessageConst.PROPERTY_UNIQ_CLIENT_MESSAGE_ID_KEYIDX));
            }
            MessageAccessor.putProperty(msg, MessageConst.PROPERTY_MIN_OFFSET,
                Long.toString(pullResult.getMinOffset()));
            MessageAccessor.putProperty(msg, MessageConst.PROPERTY_MAX_OFFSET,
                Long.toString(pullResult.getMaxOffset()));
        }

        pullResultExt.setMsgFoundList(msgListFilterAgain);
    }

    pullResultExt.setMessageBinary(null);

    return pullResult;
}

首先調用updatePullFromWhichNode方法：

public void updatePullFromWhichNode(final MessageQueue mq, final long brokerId) {
   AtomicLong suggest = this.pullFromWhichNodeTable.get(mq);
    if (null == suggest) {
        this.pullFromWhichNodeTable.put(mq, new AtomicLong(brokerId));
    } else {
        suggest.set(brokerId);
    }
}

這裏就會將pullFromWhichNodeTable中記錄的消息隊列和BrokerID的映射，更新爲Broker發送過來的建議ID
結合上一篇博客來看，若是採用集羣模式，就完成了消費者端的負載均衡

在PullStatus.FOUND情況下，會調用MessageDecoder的decodes方法，將CommitLog格式的消息數據進行解碼，轉化爲真正可讀的消息

之後會對Tag進行判斷，設置了Tag，添加Tag消息記錄

之後，在設置了FilterMessageHook鉤子情況下，通過executeHook方法執行FilterMessageHook鉤子的filterMessage方法：

public void executeHook(final FilterMessageContext context) {
    if (!this.filterMessageHookList.isEmpty()) {
        for (FilterMessageHook hook : this.filterMessageHookList) {
            try {
                hook.filterMessage(context);
            } catch (Throwable e) {
                log.error("execute hook error. hookName={}", hook.hookName());
            }
        }
    }
}

然後對消息進行屬性設置

processPullResult完成後，若是設置了ConsumeMessageHook鉤子，調用executeHookBefore和executeHookAfter方法，分別執行鉤子中的consumeMessageBefore和consumeMessageAfter方法：

public void executeHookBefore(final ConsumeMessageContext context) {
    if (!this.consumeMessageHookList.isEmpty()) {
        for (ConsumeMessageHook hook : this.consumeMessageHookList) {
            try {
                hook.consumeMessageBefore(context);
            } catch (Throwable ignored) {
            }
        }
    }
}

public void executeHookAfter(final ConsumeMessageContext context) {
    if (!this.consumeMessageHookList.isEmpty()) {
        for (ConsumeMessageHook hook : this.consumeMessageHookList) {
            try {
                hook.consumeMessageAfter(context);
            } catch (Throwable ignored) {
            }
        }
    }
}

PullConsumer消息的同步拉取到此結束

異步消息拉取

異步拉取的API都通過pullAsyncImpl方法實現：

private void pullAsyncImpl(
    final MessageQueue mq,
    final SubscriptionData subscriptionData,
    final long offset,
    final int maxNums,
    final PullCallback pullCallback,
    final boolean block,
    final long timeout) throws MQClientException, RemotingException, InterruptedException {
    this.makeSureStateOK();

    if (null == mq) {
        throw new MQClientException("mq is null", null);
    }

    if (offset < 0) {
        throw new MQClientException("offset < 0", null);
    }

    if (maxNums <= 0) {
        throw new MQClientException("maxNums <= 0", null);
    }

    if (null == pullCallback) {
        throw new MQClientException("pullCallback is null", null);
    }

    this.subscriptionAutomatically(mq.getTopic());

    try {
        int sysFlag = PullSysFlag.buildSysFlag(false, block, true, false);

        long timeoutMillis = block ? this.defaultMQPullConsumer.getConsumerTimeoutMillisWhenSuspend() : timeout;

        boolean isTagType = ExpressionType.isTagType(subscriptionData.getExpressionType());
        this.pullAPIWrapper.pullKernelImpl(
            mq,
            subscriptionData.getSubString(),
            subscriptionData.getExpressionType(),
            isTagType ? 0L : subscriptionData.getSubVersion(),
            offset,
            maxNums,
            sysFlag,
            0,
            this.defaultMQPullConsumer.getBrokerSuspendMaxTimeMillis(),
            timeoutMillis,
            CommunicationMode.ASYNC,
            new PullCallback() {

                @Override
                public void onSuccess(PullResult pullResult) {
                    pullCallback
                        .onSuccess(DefaultMQPullConsumerImpl.this.pullAPIWrapper.processPullResult(mq, pullResult, subscriptionData));
                }

                @Override
                public void onException(Throwable e) {
                    pullCallback.onException(e);
                }
            });
    } catch (MQBrokerException e) {
        throw new MQClientException("pullAsync unknow exception", e);
    }
}

相比同步，參數多了個PullCallback，用於處理異步拉取後的回調

過程基本上個同步拉取類似，只不過在調用pullKernelImpl方法時，會創建一個PullCallback
在onSuccess和onException中，實際上調用了pullCallback的相應方法，這樣就完成了異步的回調

在onSuccess回調的參數中，同同步方式類似，會通過processPullResult方法，對結果進一步加工

之後的pullKernelImpl方法和同步一樣

只不過最後調用了pullMessageAsync方法：

private void pullMessageAsync(
    final String addr,
    final RemotingCommand request,
    final long timeoutMillis,
    final PullCallback pullCallback
) throws RemotingException, InterruptedException {
    this.remotingClient.invokeAsync(addr, request, timeoutMillis, new InvokeCallback() {
        @Override
        public void operationComplete(ResponseFuture responseFuture) {
            RemotingCommand response = responseFuture.getResponseCommand();
            if (response != null) {
                try {
                    PullResult pullResult = MQClientAPIImpl.this.processPullResponse(response);
                    assert pullResult != null;
                    pullCallback.onSuccess(pullResult);
                } catch (Exception e) {
                    pullCallback.onException(e);
                }
            } else {
                if (!responseFuture.isSendRequestOK()) {
                    pullCallback.onException(new MQClientException("send request failed to " + addr + ". Request: " + request, responseFuture.getCause()));
                } else if (responseFuture.isTimeout()) {
                    pullCallback.onException(new MQClientException("wait response from " + addr + " timeout :" + responseFuture.getTimeoutMillis() + "ms" + ". Request: " + request,
                        responseFuture.getCause()));
                } else {
                    pullCallback.onException(new MQClientException("unknown reason. addr: " + addr + ", timeoutMillis: " + timeoutMillis + ". Request: " + request, responseFuture.getCause()));
                }
            }
        }
    });
}

這裏實際上也是通過Netty完成異步發送
詳見：
【RocketMQ中Producer消息的發送源碼分析】

由於是異步發送，這裏又設置了一個回調InvokeCallback
當請求發送完成，收到響應後，就會執行InvokeCallback的operationComplete方法，

在operationComplete方法中，和同步一樣，執行processPullResponse方法，處理響應
之後調用pullCallback的onSuccess方法，也就是剛纔創建的回調接口，進而執行用戶傳入的回調接口的方法

消息異步拉取也就到此結束