RabbitMQ基於Java的定時任務實現

RabbitMQ的定時任務實現主要原理是藉助rabbitmq的消息過期機制,發送消息時可以指定一個expiration(單位毫秒),當一個消息在一個隊列內過期時,在默認情況下會drop丟棄掉(此處有個條件,就是該消息必須位於隊首,也就是即將被消費時纔會判斷是否過期,也就是說不在隊首的消息即使expiration到期也無法丟棄到死信隊列)。如果隊列配置了死信隊列exchange和routing key, 則會將此到期的消息路由到routing key對應的隊列內。

    rabbitmq中對於發佈出去但是無法路由到任意一個隊列的消息會返回給publisher,如果publisher設置了
處理該情況的returnListener可以選擇如何處理,如果沒設置默認就drop丟棄掉了。
    對於設置了expiration的定時消息,到達過期時間後,默認行爲也是drop丟棄掉,如果隊列聲明時,配置了
死信處理參數,x-dead-letter-exchange和x-dead-letter-routing-key,分別指定死信交換機和死信路由
key,則在消息過期後,該消息會通過x-dead-letter-exchange設置的exchange根據x-dead-letter-routing-key
設置的routing key將消息路由到routing key綁定的queue隊列。(但是到期的消息需滿足在隊首,就是即將被消費,否則即使到期也無法被丟棄)

下面對應rabbitmq的amqp-client 5.7.2

package demo.rabbitmq.schedule;

import com.rabbitmq.client.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;
import java.net.URISyntaxException;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeoutException;

/**
 * @author ZHUFEIFEI
 */
public class Demo01 {

    private static final Logger log = LoggerFactory.getLogger(demo.rabbitmq.Demo01.class);

    private static String handleMsgQueue = "handleMsgQueue";

    private static String scheduleMsgQueue = "scheduleMsgQueue";
    //兩個隊列用的同一個exchange,也可以不同
    private static String deadLetterExchange = "deadLetterExchangeForScheduleMsg";

    private static String deadRoutingKey = "dead_routing_key";

    private static String scheduleRoutingKey = "schedule_routing_key";

    private static CountDownLatch cdl = new CountDownLatch(2);

    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException, IOException, TimeoutException, InterruptedException {
        ConnectionFactory factory = new ConnectionFactory();
//        factory.setUri("amqp://guest:guest@localhost:5672//");
        factory.setUsername("guest");
        factory.setPassword("guest");
        factory.setVirtualHost("/");
        factory.setHost("localhost");
        factory.setPort(5672);

        Connection conn = factory.newConnection();

        Channel channel = conn.createChannel();

        //聲明死信隊列, 發送的消息到死信隊列,此處死信隊列扮演的就是定時消息隊列,進入該隊列的消息需要設置expiration,該隊列不需要消費者
        Map<String, Object> arguments = new HashMap<>(2);
        //指定死信處理exchange, 死信由該exchange處理
        arguments.put("x-dead-letter-exchange", deadLetterExchange);
        //配置消息過期後,路由到哪個routing key,該key應該是對應真正的任務隊列,消費者消費真正的任務隊列
        arguments.put("x-dead-letter-routing-key", deadRoutingKey);
        AMQP.Queue.DeclareOk resultQueue = channel.queueDeclare(scheduleMsgQueue, true, false, false, arguments);
        log.info("queue declare -> {}", resultQueue);
        //聲明死信交換機
        AMQP.Exchange.DeclareOk resultExchange = channel.exchangeDeclare(deadLetterExchange, BuiltinExchangeType.DIRECT);
        log.info("exchange declare -> {}", resultExchange);
        //綁定消息隊列到一個處理器,此處理器可以和死信隊列處理器是一個也可以不是一個,此處用一個exchange
        AMQP.Queue.BindOk resultBind = channel.queueBind(scheduleMsgQueue, deadLetterExchange, scheduleRoutingKey);
        log.info("bind result -> {}", resultBind);

        //聲明真正延遲消息處理的隊列,當消息在定時消息隊列過期後會當成死信放到該隊列,消費者消費該隊列
        resultQueue = channel.queueDeclare(handleMsgQueue, true, false, false, null);
        log.info("queue declare -> {}", resultQueue);
        //爲處理消息隊列綁定exchange,並指定routing key, 該routing key就是定時隊列過期後的消息要路由的那個key
        resultBind = channel.queueBind(handleMsgQueue, deadLetterExchange, deadRoutingKey);
        log.info("bind result -> {}", resultBind);

        publishMsg(channel);
        
        consumeMsg(channel);

        cdl.await();
        channel.close();
        conn.close();
    }

    private static void consumeMsg(Channel channel) throws IOException {
        channel.basicConsume(handleMsgQueue, false, "myScheduleConsumerTag",
                new DefaultConsumer(channel) {
                    @Override
                    public void handleDelivery(String consumerTag,
                                               Envelope envelope,
                                               AMQP.BasicProperties properties,
                                               byte[] body)
                            throws IOException {
                        String routingKey = envelope.getRoutingKey();
                        String contentType = properties.getContentType();
                        long deliveryTag = envelope.getDeliveryTag();

                        log.info("consumerTag -> {}, routingKey -> {}, contentType -> {}, deliveryTag -> {}, content -> {}"
                                , consumerTag, routingKey
                                , contentType, deliveryTag
                                , new String(body)
                                );

                        log.info("basic properties -> {}", properties);
                        channel.basicAck(deliveryTag, false);

                        cdl.countDown();
                    }
                });
    }

    private static void publishMsg(Channel channel) throws IOException {
        byte[] messageBodyBytes = "Hello, world!".getBytes();
        //發送消息,並自定義屬性, 同上效果
        channel.basicPublish(deadLetterExchange, scheduleRoutingKey,
                new AMQP.BasicProperties.Builder()
                        .contentType("text/plain")
                        .deliveryMode(2)
                        .priority(1)
                        .expiration("30000") // 30秒
                        .build(),
                messageBodyBytes);
        channel.basicPublish(deadLetterExchange, scheduleRoutingKey,
                new AMQP.BasicProperties.Builder()
                        .contentType("text/plain")
                        .deliveryMode(2)
                        .priority(1)
                        .expiration("60000") // 60秒
                        .build(),
                messageBodyBytes);
        log.info("message published!");
    }
}

        還有另一種方式的定時任務實現,rabbitmq插件社區提供了rabbitmq_delayed_message_exchange插件,提供定時 消息支持,內部採用erlang定時器定時檢查消息的x-delay屬性是否在0-(2^32-1)區間內,x-delay是發送消息時的一個屬性,毫秒過期值,當該值小於0時,就需要發送到消息隊列。

       插件形式的定是消息支持侷限性較多,默認採用disc節點方式存儲並且在當前節點只有一個副本,消息只是延遲路由到指定的消息隊列,在路由到隊列前存儲在mnesia數據表中,會有其他邏輯來檢測消息是否到期。該插件提供的exchange類型爲x-delayed-message, 是默認的4種exchange的代理,內部路由邏輯還是使用的內置的4個exchange,由於多了層代理並且需要erlang定時器定時檢測,因此性能上要差一些,並且數據量不宜過大(最好不要超過十萬量級),不能嚴格保證是x-delay的時間後準時執行,插件內的定時器不會持久化,會在每次插件重啓時初始化,一旦禁用插件,所有待發布的定時消息將會丟失。

 

注意:基於死信隊列的實現有一個問題,就是消息隊列對於每個消息帶有ttl的情況,不會定時掃描隊列檢查消息是否過期,而是在消費者消費到某一個消息時(或者說某個消息到達隊首時),對該消息的expiration進行判斷,如果過期直接丟到死信隊列。因此如果定時隊列沒有消費者,並且消息的過期時間參差不齊(不是有序的,就是後來的消息過期時間可能小於之前到達的消息),則消息無法定時被丟到死信隊列中去!

 

參考:scheduling-messages-with-rabbitmq

參考:rabbitmq-delayed-message-exchange

參考:message/queue ttl

 

 

 

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章