消息100%的投遞
消息如何保障100%的投遞成功?
什麼是生產端的可靠性投遞?
- 保障消息的成功發出
- 保障MQ節點的成功接收
- 發送端收到MQ節點(Broker)確認應答
- 完善的消息進行補償機制
BAT/TMD互聯網大廠的解決方案:
- 消息落庫,對消息狀態進行打標
- 消息的延遲投遞,做二次確認,回調檢查
冪等性概念
冪等性是什麼?
- 我們可以借鑑數據庫的樂觀鎖機制
- 比如我們執行一條更新庫存的SQL語句
- Update t_repository set count = count -1,version = version + 1 where version = 1
- Elasticsearch也是嚴格遵循冪等性概念,每次數據更新,version+1(博主博客前面有提到)
消費端-冪等性保障
在海量訂單產生的業務高峯期,如何避免消息的重複消費問題?
消費實現冪等性,就意味着,我們的消息永遠不會消費多次,即使我們收到了多條一樣的消息
業界主流的冪等性操作
唯一ID+指紋碼機制,利用數據庫主鍵去重
利用Redis的原子性去實現
唯一ID+指紋碼 機制
唯一ID+指紋碼機制,利用數據庫主鍵去重
Select count(1) from T_order where ID=唯一ID+指紋碼
好處:實現簡單
壞處:高併發下有數據庫寫入的性能瓶頸
解決方案:根據ID進行分庫分表進行算法路由
利用Redis的原子性去實現
使用Redis進行冪等,需要考慮的問題
第一:我們是否要進行數據落庫,如果落庫的話,關鍵解決的問題是數據庫和緩存如何做到原子性?
第二:如果不進行落庫,那麼都存儲到緩存中,如何設置定時同步策略?
Confirm確認消息
理解Confirm消息確認機制
消息的確認,是指生產者投遞消息後,如果Broker收到消息,則會給我們生產者 一個應答。
生產者進行接收應答,用來確定這條消息是否正常的發送到Broker,這種方式也是消息的可靠性投遞的核心保障
如何實現Confirm確認消息?
第一步:在Channel上開啓確認模式:channel.confirmSelect()
第二步:在channel上添加監聽:addConfirmListener,監聽成功和失敗的返回結果,根據具體的結果對消息進行重新發送、或記錄日誌等後續處理!
消費端代碼:
package com.zh.rabbitmqapi.confirm;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.QueueingConsumer;
public class Consumer {
public static void main(String[] args) throws Exception {
//1 創建ConnectionFactory
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.43.35");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
//2 獲取C onnection
Connection connection = connectionFactory.newConnection();
//3 通過Connection創建一個新的Channel
Channel channel = connection.createChannel();
String exchangeName = "test_confirm_exchange";
String routingKey = "confirm.#";
String queueName = "test_confirm_queue";
//4 聲明交換機和隊列 然後進行綁定設置, 最後制定路由Key
channel.exchangeDeclare(exchangeName, "topic", true);
channel.queueDeclare(queueName, true, false, false, null);
channel.queueBind(queueName, exchangeName, routingKey);
//5 創建消費者
QueueingConsumer queueingConsumer = new QueueingConsumer(channel);
channel.basicConsume(queueName, true, queueingConsumer);
while(true){
QueueingConsumer.Delivery delivery = queueingConsumer.nextDelivery();
String msg = new String(delivery.getBody());
System.err.println("消費端: " + msg);
}
}
}
服務提供方代碼:
package com.zh.rabbitmqapi.confirm;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.ConfirmListener;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import java.io.IOException;
public class Producer {
public static void main(String[] args) throws Exception {
//1 創建ConnectionFactory
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.43.35");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
//2 獲取C onnection
Connection connection = connectionFactory.newConnection();
//3 通過Connection創建一個新的Channel
Channel channel = connection.createChannel();
//4 指定我們的消息投遞模式: 消息的確認模式
channel.confirmSelect();
String exchangeName = "test_confirm_exchange";
String routingKey = "confirm.save";
//5 發送一條消息
String msg = "Hello RabbitMQ Send confirm message!";
channel.basicPublish(exchangeName, routingKey, null, msg.getBytes());
//6 添加一個確認監聽
channel.addConfirmListener(new ConfirmListener() {
@Override
public void handleNack(long deliveryTag, boolean multiple) throws IOException {
System.err.println("-------no ack!-----------");
}
@Override
public void handleAck(long deliveryTag, boolean multiple) throws IOException {
System.err.println("-------ack!-----------");
}
});
}
}
Return返回消息
Return Listener用於處理一些不可路由的消息!
正常情況:我們的消息生產者,通過指定一個Exchange和RoutingKey,把消息送達到某一個隊列中去,然後我們的消費者監聽隊列,進行消費處理操作!
異常情況:在某些情況下,如果我們在發送消息的時候,當前的Exchange不存在或者指定的路由key路由不到,這個時候如果我們需要監聽這種不可達的消息,就需要使用Return Listener!
在基礎API中有一個關鍵的配置項
Mandatory:如果爲true,則監聽器會接收到路由不可達的消息,然後進行後續處理,如果爲false,那麼Broker端自動刪除該消息!
消費端代碼:
package com.zh.rabbitmqapi.returnlistener;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.QueueingConsumer;
public class Consumer {
public static void main(String[] args) throws Exception {
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.43.35");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
Connection connection = connectionFactory.newConnection();
Channel channel = connection.createChannel();
String exchangeName = "test_return_exchange";
String routingKey = "return.#";
String queueName = "test_return_queue";
channel.exchangeDeclare(exchangeName, "topic", true, false, null);
channel.queueDeclare(queueName, true, false, false, null);
channel.queueBind(queueName, exchangeName, routingKey);
QueueingConsumer queueingConsumer = new QueueingConsumer(channel);
channel.basicConsume(queueName, true, queueingConsumer);
while(true){
QueueingConsumer.Delivery delivery = queueingConsumer.nextDelivery();
String msg = new String(delivery.getBody());
System.err.println("消費者: " + msg);
}
}
}
生產端代碼:
package com.zh.rabbitmqapi.returnlistener;
import com.rabbitmq.client.*;
import java.io.IOException;
public class Producer {
public static void main(String[] args) throws Exception {
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.43.35");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
Connection connection = connectionFactory.newConnection();
Channel channel = connection.createChannel();
String exchange = "test_return_exchange";
String routingKey = "return.save";
String routingKeyError = "abc.save";
String msg = "Hello RabbitMQ Return Message";
channel.addReturnListener(new ReturnListener() {
@Override
public void handleReturn(int replyCode, String replyText, String exchange,
String routingKey, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.err.println("---------handle return----------");
System.err.println("replyCode: " + replyCode);
System.err.println("replyText: " + replyText);
System.err.println("exchange: " + exchange);
System.err.println("routingKey: " + routingKey);
System.err.println("properties: " + properties);
System.err.println("body: " + new String(body));
}
});
//消息投遞成功,會被消費者所消費
// channel.basicPublish(exchange, routingKey, true, null, msg.getBytes());
//消息不可達,將觸發ReturnListener
channel.basicPublish(exchange, routingKeyError, true, null, msg.getBytes());
}
}
投遞不成功:
消息投遞成功:
自定義消費者
我們一般就是在代碼中編寫while循環,進行consumer.nextDelivery方法進行獲取下一條消息,然後進行消費處理!
但是我們使用自定義的Consumer更加的方便,解耦性更加的強,也是實際工作中最常用的使用方式!
自定義消費端代碼
package com.zh.rabbitmqapi.consumer;
import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DefaultConsumer;
import com.rabbitmq.client.Envelope;
import java.io.IOException;
public class MyConsumer extends DefaultConsumer {
public MyConsumer(Channel channel) {
super(channel);
}
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.err.println("-----------consume message----------");
System.err.println("consumerTag: " + consumerTag);
System.err.println("envelope: " + envelope);
System.err.println("properties: " + properties);
System.err.println("body: " + new String(body));
}
}
消費端調用:
package com.zh.rabbitmqapi.consumer;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
public class Consumer {
public static void main(String[] args) throws Exception {
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.43.35");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
Connection connection = connectionFactory.newConnection();
Channel channel = connection.createChannel();
String exchangeName = "test_consumer_exchange";
String routingKey = "consumer.#";
String queueName = "test_consumer_queue";
channel.exchangeDeclare(exchangeName, "topic", true, false, null);
channel.queueDeclare(queueName, true, false, false, null);
channel.queueBind(queueName, exchangeName, routingKey);
channel.basicConsume(queueName, true, new MyConsumer(channel));
}
}
生產端調用:
package com.zh.rabbitmqapi.consumer;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
public class Producer {
public static void main(String[] args) throws Exception {
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.43.35");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
Connection connection = connectionFactory.newConnection();
Channel channel = connection.createChannel();
String exchange = "test_consumer_exchange";
String routingKey = "consumer.save";
String msg = "Hello RabbitMQ Consumer Message";
for(int i =0; i<5; i ++){
channel.basicPublish(exchange, routingKey, true, null, msg.getBytes());
}
}
}