RabbitMQ的定時任務實現主要原理是藉助rabbitmq的消息過期機制,發送消息時可以指定一個expiration(單位毫秒),當一個消息在一個隊列內過期時,在默認情況下會drop丟棄掉(此處有個條件,就是該消息必須位於隊首,也就是即將被消費時纔會判斷是否過期,也就是說不在隊首的消息即使expiration到期也無法丟棄到死信隊列)。如果隊列配置了死信隊列exchange和routing key, 則會將此到期的消息路由到routing key對應的隊列內。
rabbitmq中對於發佈出去但是無法路由到任意一個隊列的消息會返回給publisher,如果publisher設置了
處理該情況的returnListener可以選擇如何處理,如果沒設置默認就drop丟棄掉了。
對於設置了expiration的定時消息,到達過期時間後,默認行爲也是drop丟棄掉,如果隊列聲明時,配置了
死信處理參數,x-dead-letter-exchange和x-dead-letter-routing-key,分別指定死信交換機和死信路由
key,則在消息過期後,該消息會通過x-dead-letter-exchange設置的exchange根據x-dead-letter-routing-key
設置的routing key將消息路由到routing key綁定的queue隊列。(但是到期的消息需滿足在隊首,就是即將被消費,否則即使到期也無法被丟棄)
下面對應rabbitmq的amqp-client 5.7.2
package demo.rabbitmq.schedule;
import com.rabbitmq.client.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.net.URISyntaxException;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeoutException;
/**
* @author ZHUFEIFEI
*/
public class Demo01 {
private static final Logger log = LoggerFactory.getLogger(demo.rabbitmq.Demo01.class);
private static String handleMsgQueue = "handleMsgQueue";
private static String scheduleMsgQueue = "scheduleMsgQueue";
//兩個隊列用的同一個exchange,也可以不同
private static String deadLetterExchange = "deadLetterExchangeForScheduleMsg";
private static String deadRoutingKey = "dead_routing_key";
private static String scheduleRoutingKey = "schedule_routing_key";
private static CountDownLatch cdl = new CountDownLatch(2);
public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException, IOException, TimeoutException, InterruptedException {
ConnectionFactory factory = new ConnectionFactory();
// factory.setUri("amqp://guest:guest@localhost:5672//");
factory.setUsername("guest");
factory.setPassword("guest");
factory.setVirtualHost("/");
factory.setHost("localhost");
factory.setPort(5672);
Connection conn = factory.newConnection();
Channel channel = conn.createChannel();
//聲明死信隊列, 發送的消息到死信隊列,此處死信隊列扮演的就是定時消息隊列,進入該隊列的消息需要設置expiration,該隊列不需要消費者
Map<String, Object> arguments = new HashMap<>(2);
//指定死信處理exchange, 死信由該exchange處理
arguments.put("x-dead-letter-exchange", deadLetterExchange);
//配置消息過期後,路由到哪個routing key,該key應該是對應真正的任務隊列,消費者消費真正的任務隊列
arguments.put("x-dead-letter-routing-key", deadRoutingKey);
AMQP.Queue.DeclareOk resultQueue = channel.queueDeclare(scheduleMsgQueue, true, false, false, arguments);
log.info("queue declare -> {}", resultQueue);
//聲明死信交換機
AMQP.Exchange.DeclareOk resultExchange = channel.exchangeDeclare(deadLetterExchange, BuiltinExchangeType.DIRECT);
log.info("exchange declare -> {}", resultExchange);
//綁定消息隊列到一個處理器,此處理器可以和死信隊列處理器是一個也可以不是一個,此處用一個exchange
AMQP.Queue.BindOk resultBind = channel.queueBind(scheduleMsgQueue, deadLetterExchange, scheduleRoutingKey);
log.info("bind result -> {}", resultBind);
//聲明真正延遲消息處理的隊列,當消息在定時消息隊列過期後會當成死信放到該隊列,消費者消費該隊列
resultQueue = channel.queueDeclare(handleMsgQueue, true, false, false, null);
log.info("queue declare -> {}", resultQueue);
//爲處理消息隊列綁定exchange,並指定routing key, 該routing key就是定時隊列過期後的消息要路由的那個key
resultBind = channel.queueBind(handleMsgQueue, deadLetterExchange, deadRoutingKey);
log.info("bind result -> {}", resultBind);
publishMsg(channel);
consumeMsg(channel);
cdl.await();
channel.close();
conn.close();
}
private static void consumeMsg(Channel channel) throws IOException {
channel.basicConsume(handleMsgQueue, false, "myScheduleConsumerTag",
new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag,
Envelope envelope,
AMQP.BasicProperties properties,
byte[] body)
throws IOException {
String routingKey = envelope.getRoutingKey();
String contentType = properties.getContentType();
long deliveryTag = envelope.getDeliveryTag();
log.info("consumerTag -> {}, routingKey -> {}, contentType -> {}, deliveryTag -> {}, content -> {}"
, consumerTag, routingKey
, contentType, deliveryTag
, new String(body)
);
log.info("basic properties -> {}", properties);
channel.basicAck(deliveryTag, false);
cdl.countDown();
}
});
}
private static void publishMsg(Channel channel) throws IOException {
byte[] messageBodyBytes = "Hello, world!".getBytes();
//發送消息,並自定義屬性, 同上效果
channel.basicPublish(deadLetterExchange, scheduleRoutingKey,
new AMQP.BasicProperties.Builder()
.contentType("text/plain")
.deliveryMode(2)
.priority(1)
.expiration("30000") // 30秒
.build(),
messageBodyBytes);
channel.basicPublish(deadLetterExchange, scheduleRoutingKey,
new AMQP.BasicProperties.Builder()
.contentType("text/plain")
.deliveryMode(2)
.priority(1)
.expiration("60000") // 60秒
.build(),
messageBodyBytes);
log.info("message published!");
}
}
還有另一種方式的定時任務實現,rabbitmq插件社區提供了rabbitmq_delayed_message_exchange
插件,提供定時 消息支持,內部採用erlang定時器定時檢查消息的x-delay屬性是否在0-(2^32-1)區間內,x-delay是發送消息時的一個屬性,毫秒過期值,當該值小於0時,就需要發送到消息隊列。
插件形式的定是消息支持侷限性較多,默認採用disc節點方式存儲並且在當前節點只有一個副本,消息只是延遲路由到指定的消息隊列,在路由到隊列前存儲在mnesia數據表中,會有其他邏輯來檢測消息是否到期。該插件提供的exchange類型爲x-delayed-message, 是默認的4種exchange的代理,內部路由邏輯還是使用的內置的4個exchange,由於多了層代理並且需要erlang定時器定時檢測,因此性能上要差一些,並且數據量不宜過大(最好不要超過十萬量級),不能嚴格保證是x-delay的時間後準時執行,插件內的定時器不會持久化,會在每次插件重啓時初始化,一旦禁用插件,所有待發布的定時消息將會丟失。
注意:基於死信隊列的實現有一個問題,就是消息隊列對於每個消息帶有ttl的情況,不會定時掃描隊列檢查消息是否過期,而是在消費者消費到某一個消息時(或者說某個消息到達隊首時),對該消息的expiration進行判斷,如果過期直接丟到死信隊列。因此如果定時隊列沒有消費者,並且消息的過期時間參差不齊(不是有序的,就是後來的消息過期時間可能小於之前到達的消息),則消息無法定時被丟到死信隊列中去!
參考:scheduling-messages-with-rabbitmq
參考:rabbitmq-delayed-message-exchange