本文主要分析RocketMQ中如何保證消息有序的。
RocketMQ的版本爲:4.2.0 release。
一.時序圖
還是老規矩,先把分析過程的時序圖擺出來:
1.Producer發送順序消息
2.Consumer接收順序消息(一)
3.Consumer接收順序消息(二)
二.源碼分析 - Producer發送順序消息
1 DefaultMQProducer#send:發送消息,入參中有自定義的消息隊列選擇器。
// DefaultMQProducer#send
public SendResult send(Message msg, MessageQueueSelector selector, Object arg)
throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
return this.defaultMQProducerImpl.send(msg, selector, arg);
}
1.1 DefaultMQProducerImpl#makeSureStateOK:確保Producer的狀態是運行狀態-ServiceState.RUNNING。
// DefaultMQProducerImpl#makeSureStateOK
private void makeSureStateOK() throws MQClientException {
if (this.serviceState != ServiceState.RUNNING) {
throw new MQClientException("The producer service state not OK, "+ this.serviceState
+ FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
null);
}
}
1.2 DefaultMQProducerImpl#tryToFindTopicPublishInfo:根據Topic獲取發佈Topic用到的路由信息。
// DefaultMQProducerImpl#tryToFindTopicPublishInfo
private TopicPublishInfo tryToFindTopicPublishInfo(final String topic) {
TopicPublishInfo topicPublishInfo = this.topicPublishInfoTable.get(topic);
if (null == topicPublishInfo || !topicPublishInfo.ok()) {
this.topicPublishInfoTable.putIfAbsent(topic, new TopicPublishInfo());
this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic);// 爲空則從 NameServer更新獲取,false,不傳入 defaultMQProducer
topicPublishInfo = this.topicPublishInfoTable.get(topic);
}
if (topicPublishInfo.isHaveTopicRouterInfo() || topicPublishInfo.ok()) {// 有了路由信息而且狀態OK,則返回
return topicPublishInfo;
} else {
this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic, true, this.defaultMQProducer);
topicPublishInfo = this.topicPublishInfoTable.get(topic);
return topicPublishInfo;
}
}
1.3 調用自定義消息隊列選擇器的select方法。
// DefaultMQProducerImpl#sendSelectImpl
MessageQueue mq = null;
try {
mq = selector.select(topicPublishInfo.getMessageQueueList(), msg, arg);
} catch (Throwable e) {
throw new MQClientException("select message queue throwed exception.", e);
}
// Producer#main
SendResult sendResult = producer.send(msg, new MessageQueueSelector() {
@Override
public MessageQueue select(List<MessageQueue> mqs, Message msg, Object arg) {
Integer id = (Integer) arg;
int index = id % mqs.size();
return mqs.get(index);
}
}, orderId);
1.4 DefaultMQProducerImpl#sendKernelImpl:發送消息的核心實現方法。
// DefaultMQProducerImpl#sendKernelImpl
......
switch (communicationMode) {
case SYNC:
long costTimeSync = System.currentTimeMillis() - beginStartTime;
if (timeout < costTimeSync) {
throw new RemotingTooMuchRequestException("sendKernelImpl call timeout");
}
sendResult = this.mQClientFactory.getMQClientAPIImpl().sendMessage(
brokerAddr,
mq.getBrokerName(),
msg,
requestHeader,
timeout - costTimeSync,
communicationMode,
context,
this);
break;
......
1.4.1 MQClientAPIImpl#sendMessage:發送消息。
// MQClientAPIImpl#sendMessage
......
switch (communicationMode) {// 根據發送消息的模式(同步/異步)選擇不同的方式,默認是同步
case SYNC:
long costTimeSync = System.currentTimeMillis() - beginStartTime;
if (timeoutMillis < costTimeSync) {
throw new RemotingTooMuchRequestException("sendMessage call timeout");
}
return this.sendMessageSync(addr, brokerName, msg, timeoutMillis - costTimeSync, request);
......
1.4.1.1 MQClientAPIImpl#sendMessageSync:發送同步消息。
// MQClientAPIImpl#sendMessageSync
private SendResult sendMessageSync(
final String addr,
final String brokerName,
final Message msg,
final long timeoutMillis,
final RemotingCommand request
) throws RemotingException, MQBrokerException, InterruptedException {
RemotingCommand response = this.remotingClient.invokeSync(addr, request, timeoutMillis);
assert response != null;
return this.processSendResponse(brokerName, msg, response);
}
1.4.1.1.1 NettyRemotingClient#invokeSync:構造RemotingCommand,調用的方式是同步。
// NettyRemotingClient#invokeSync
RemotingCommand response = this.invokeSyncImpl(channel, request, timeoutMillis - costTime);
if (this.rpcHook != null) {
this.rpcHook.doAfterResponse(RemotingHelper.parseChannelRemoteAddr(channel), request, response);
}
return response;
三.源碼分析 - Consumer接收順序消息(一)
1 DefaultMQPushConsumer#registerMessageListener:把Consumer傳入的消息監聽器加入到messageListener中。
// DefaultMQPushConsumer#registerMessageListener
public void registerMessageListener(MessageListenerOrderly messageListener) {
this.messageListener = messageListener;
this.defaultMQPushConsumerImpl.registerMessageListener(messageListener);
}
1.1 DefaultMQPushConsumerImpl#registerMessageListener:把Consumer傳入的消息監聽器加入到messageListenerInner中。
// DefaultMQPushConsumerImpl#registerMessageListener
public void registerMessageListener(MessageListener messageListener) {
this.messageListenerInner = messageListener;
}
2 DefaultMQPushConsumer#start:啓動Consumer。
// DefaultMQPushConsumer#start
public void start() throws MQClientException {
this.defaultMQPushConsumerImpl.start();
}
2.1 DefaultMQPushConsumerImpl#start:啓動ConsumerImpl。
// DefaultMQPushConsumerImpl#start
switch (this.serviceState) {
case CREATE_JUST:// 剛剛創建
......
if (this.getMessageListenerInner() instanceof MessageListenerOrderly) {// 有序消息服務
this.consumeOrderly = true;
this.consumeMessageService = new ConsumeMessageOrderlyService(this, (MessageListenerOrderly) this.getMessageListenerInner());
} else if (this.getMessageListenerInner() instanceof MessageListenerConcurrently) {// 併發無序消息服務
this.consumeOrderly = false;
this.consumeMessageService = new ConsumeMessageConcurrentlyService(this, (MessageListenerConcurrently) this.getMessageListenerInner());
}
......
this.consumeMessageService.start();// 啓動消息服務
......
mQClientFactory.start();// 啓動MQClientInstance
......
2.1.1 new ConsumeMessageOrderlyService():構造順序消息服務。
// ConsumeMessageOrderlyService#ConsumeMessageOrderlyService
public ConsumeMessageOrderlyService(DefaultMQPushConsumerImpl defaultMQPushConsumerImpl,
MessageListenerOrderly messageListener) {
this.defaultMQPushConsumerImpl = defaultMQPushConsumerImpl;
this.messageListener = messageListener;
this.defaultMQPushConsumer = this.defaultMQPushConsumerImpl.getDefaultMQPushConsumer();
this.consumerGroup = this.defaultMQPushConsumer.getConsumerGroup();
this.consumeRequestQueue = new LinkedBlockingQueue<Runnable>();
this.consumeExecutor = new ThreadPoolExecutor(// 主消息消費線程池,正常執行收到的ConsumeRequest。多線程
this.defaultMQPushConsumer.getConsumeThreadMin(),
this.defaultMQPushConsumer.getConsumeThreadMax(),
1000 * 60,
TimeUnit.MILLISECONDS,
this.consumeRequestQueue,
new ThreadFactoryImpl("ConsumeMessageThread_"));
this.scheduledExecutorService = Executors.newSingleThreadScheduledExecutor(new ThreadFactoryImpl("ConsumeMessageScheduledThread_"));
}
2.1.2 ConsumeMessageOrderlyService#start:啓動消息隊列客戶端實例。
// DefaultMQPushConsumerImpl#start
this.consumeMessageService.start();
// ConsumeMessageOrderlyService#start
public void start() {
if (MessageModel.CLUSTERING.equals(ConsumeMessageOrderlyService.this.defaultMQPushConsumerImpl.messageModel())) {
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
ConsumeMessageOrderlyService.this.lockMQPeriodically();// 定時向broker發送批量鎖住當前正在消費的隊列集合的消息
}
}, 1000 * 1, ProcessQueue.REBALANCE_LOCK_INTERVAL, TimeUnit.MILLISECONDS);
}
}
2.1.2.1 ConsumeMessageOrderlyService#lockMQPeriodically:定時向broker發送批量鎖住當前正在消費的隊列集合的消息。
2.1.2.1.1 RebalanceImpl#lockAll:鎖住所有正在消息的隊列。
// ConsumeMessageOrderlyService#lockMQPeriodically
if (!this.stopped) {
this.defaultMQPushConsumerImpl.getRebalanceImpl().lockAll();
}
// RebalanceImpl#lockAll
HashMap<String, Set<MessageQueue>> brokerMqs = this.buildProcessQueueTableByBrokerName();// 根據brokerName從processQueueTable獲取正在消費的隊列集合
......
Set<MessageQueue> lockOKMQSet = this.mQClientFactory.getMQClientAPIImpl().lockBatchMQ(findBrokerResult.getBrokerAddr(), requestBody, 1000);// 向Broker發送鎖住消息隊列的指令
for (MessageQueue mq : lockOKMQSet) {
ProcessQueue processQueue = this.processQueueTable.get(mq);
if (processQueue != null) {
if (!processQueue.isLocked()) {
log.info("the message queue locked OK, Group: {} {}", this.consumerGroup, mq);
}
processQueue.setLocked(true);
processQueue.setLastLockTimestamp(System.currentTimeMillis());
}
}
......
2.1.3 MQClientInstance#start:啓動MQClientInstance。過程較複雜,放到大標題四中分析。
// DefaultMQPushConsumerImpl#start
mQClientFactory.start();
四.源碼分析 - Consumer接收順序消息(二)
1 MQClientInstance#start:啓動客戶端實例MQClientInstance。
// MQClientInstance#start
synchronized (this) {
switch (this.serviceState) {
case CREATE_JUST:
......
// Start pull service 啓動拉取消息服務
this.pullMessageService.start();
// Start rebalance service 啓動消費端負載均衡服務
this.rebalanceService.start();
......
1.1 PullMessageService#run:啓動拉取消息服務。實際調用的是DefaultMQPushConsumerImpl的pullMessage方法。
// PullMessageService#run
public void run() {
log.info(this.getServiceName() + " service started");
while (!this.isStopped()) {
try {
PullRequest pullRequest = this.pullRequestQueue.take();
this.pullMessage(pullRequest);
} catch (InterruptedException ignored) {
} catch (Exception e) {
log.error("Pull Message Service Run Method exception", e);
}
}
log.info(this.getServiceName() + " service end");
}
// PullMessageService#pullMessage
private void pullMessage(final PullRequest pullRequest) {
final MQConsumerInner consumer = this.mQClientFactory.selectConsumer(pullRequest.getConsumerGroup());
if (consumer != null) {
DefaultMQPushConsumerImpl impl = (DefaultMQPushConsumerImpl) consumer;
impl.pullMessage(pullRequest);// 調用DefaultMQPushConsumerImpl的pullMessage
} else {
log.warn("No matched consumer for the PullRequest {}, drop it", pullRequest);
}
}
1.1.1.1 DefaultMQPushConsumerImpl#pullMessage:拉取消息。提交到ConsumeMessageOrderlyService的線程池consumeExecutor中執行。
// DefaultMQPushConsumerImpl#pullMessage
......
PullCallback pullCallback = new PullCallback() {
@Override
public void onSuccess(PullResult pullResult) {
switch (pullResult.getPullStatus()) {
case FOUND:
long prevRequestOffset = pullRequest.getNextOffset();
pullRequest.setNextOffset(pullResult.getNextBeginOffset());
long pullRT = System.currentTimeMillis() - beginTimestamp;
......
DefaultMQPushConsumerImpl.this.consumeMessageService.submitConsumeRequest(
pullResult.getMsgFoundList(),
processQueue,
pullRequest.getMessageQueue(),
dispatchToConsume);
......
1.1.1.1.1.1.1 ConsumeRequest#run:處理消息消費的線程。
// ConsumeMessageOrderlyService.ConsumeRequest#run
List<MessageExt> msgs = this.processQueue.takeMessags(consumeBatchSize);
......
long beginTimestamp = System.currentTimeMillis();
ConsumeReturnType returnType = ConsumeReturnType.SUCCESS;
boolean hasException = false;
try {
this.processQueue.getLockConsume().lock();
if (this.processQueue.isDropped()) {
log.warn("consumeMessage, the message queue not be able to consume, because it's dropped. {}",this.messageQueue);
break;
}
status = messageListener.consumeMessage(Collections.unmodifiableList(msgs), context);// 實際消費消息的地方,回調消息監聽器的consumeMessage方法
} catch (Throwable e) {
log.warn("consumeMessage exception: {} Group: {} Msgs: {} MQ: {}",
RemotingHelper.exceptionSimpleDesc(e),
ConsumeMessageOrderlyService.this.consumerGroup,
msgs,messageQueue);
hasException = true;
} finally {
this.processQueue.getLockConsume().unlock();
}
......
1.2 RebalanceService#run:啓動消息端負載均衡服務。
// RebalanceService#run
public void run() {
log.info(this.getServiceName() + " service started");
while (!this.isStopped()) {
this.waitForRunning(waitInterval);
this.mqClientFactory.doRebalance();
}
log.info(this.getServiceName() + " service end");
}
// MQClientInstance#doRebalance
public void doRebalance() {
for (Map.Entry<String, MQConsumerInner> entry : this.consumerTable.entrySet()) {
MQConsumerInner impl = entry.getValue();
if (impl != null) {
try {
impl.doRebalance();
} catch (Throwable e) {
log.error("doRebalance exception", e);
}
}
}
}
// DefaultMQPushConsumerImpl#doRebalance
public void doRebalance() {
if (!this.pause) {
this.rebalanceImpl.doRebalance(this.isConsumeOrderly());
}
}
1.2.1.1.1 RebalanceImpl#doRebalance:負載均衡服務類處理。
// RebalanceImpl#doRebalance
public void doRebalance(final boolean isOrder) {
Map<String, SubscriptionData> subTable = this.getSubscriptionInner();
if (subTable != null) {
for (final Map.Entry<String, SubscriptionData> entry : subTable.entrySet()) {
final String topic = entry.getKey();
try {
this.rebalanceByTopic(topic, isOrder);
} catch (Throwable e) {
if (!topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
log.warn("rebalanceByTopic Exception", e);
}
}
}
}
this.truncateMessageQueueNotMyTopic();
}
// RebalanceImpl#rebalanceByTopic
switch (messageModel) {
case BROADCASTING: {
Set<MessageQueue> mqSet = this.topicSubscribeInfoTable.get(topic);
if (mqSet != null) {
boolean changed = this.updateProcessQueueTableInRebalance(topic, mqSet, isOrder);// 根據Toipc去除queue
if (changed) {
this.messageQueueChanged(topic, mqSet, mqSet);
log.info("messageQueueChanged {} {} {} {}",
consumerGroup,
topic,
mqSet,
mqSet);
}
} else {
......
// RebalanceImpl#updateProcessQueueTableInRebalance
this.dispatchPullRequest(pullRequestList);// RebalancePushImpl分發消息
1.2.1.1.1.1.1.1 RebalancePushImpl#dispatchPullRequest:RebalancePushImpl分發。
// RebalancePushImpl#dispatchPullRequest
public void dispatchPullRequest(List<PullRequest> pullRequestList) {
for (PullRequest pullRequest : pullRequestList) {
this.defaultMQPushConsumerImpl.executePullRequestImmediately(pullRequest);
log.info("doRebalance, {}, add a new pull request {}", consumerGroup, pullRequest);
}
}
五.總結
相比Producer的發送流程,Consumer的接收流程稍微複雜一點。通過上面的源碼分析,可以知道RocketMQ是怎樣保證消息的有序的:
1.通過ReblanceImp的lockAll方法,每隔一段時間定時鎖住當前消費端正在消費的隊列。設置本地隊列ProcessQueue的locked屬性爲true。保證broker中的每個消息隊列只對應一個消費端;
2.另外,消費端也是通過鎖,保證每個ProcessQueue只有一個線程消費。