RabbitMQ根據exchange的類型規則將消息路由到相應的queue,其中exchange的類型包括direct,topic,fanout以及headers,我們這裏主要介紹topic exchange類型的消息路由原理,其他exchange的路由原理相對簡單。Topic exchange的示意圖如下:
- *(星號):可以(只能)匹配一個單詞。
- #(井號):可以匹配多個單詞(或者0個)。
這裏的使用原理不做介紹,主要介紹RabbitMQ源碼內部的實現,即exchange和queue如何進行binding的。
1. exchange與queue的binding
RabbitMQ的exchange和queue的binding代碼流程如下:
對於RabbitMQ的使用流程一般爲:
- RabbitMQ客戶端與RabbitMQ服務端建立連接。
- 建立channel。
- 在建立的channel創建exchange和queue。
- 將exchange和queue進行binding。
- 對於生產者而已,向exchange發送消息,且需要指定相應的routing_key。
- 對於消費者而已,消費處理監聽的queue中的消息。
其中exchange與queue的binding便位於第4個步驟,入口代碼如下:
%% rabbit_channel.erl
%% 處理將交換機exchange和隊列進行綁定的消息
handle_method(#'queue.bind'{queue = QueueNameBin,
exchange = ExchangeNameBin,
routing_key = RoutingKey,
nowait = NoWait,
arguments = Arguments}, _, State) ->
binding_action(fun rabbit_binding:add/2,
ExchangeNameBin, queue, QueueNameBin, RoutingKey, Arguments,
#'queue.bind_ok'{}, NoWait, State);
在binding_action函數的一些合法性校驗之後,執行真正的binding函數rabbit_binding:add/2。
%% rabbit_binding.erl
%% 增加新的綁定
add(Binding, InnerFun) ->
binding_action(
Binding,
%% Src,Dst都是從mnesia數據庫中讀取到的數據
fun (Src, Dst, B) ->
%% 找到對應交換機的處理模塊去驗證綁定的合法性(Src必須是exchange交換機類型)
case rabbit_exchange:validate_binding(Src, B) of
ok ->
%% this argument is used to check queue exclusivity(排他性);
%% in general, we want to fail on that in preference to
%% anything else
%% 功效一:檢查隊列字段exclusive_owner字段的正確性,檢查排他性隊列(如果一個隊列被聲明爲排他隊列,該隊列僅對首次聲明它的連接可見)
case InnerFun(Src, Dst) of
ok ->
%% 查看rabbit_route表中是否有B這個數據,如果沒有則添加到mnesia數據庫表中
case mnesia:read({rabbit_route, B}) of
%% 增加綁定的實際操作(實際操作mnesia數據庫表的函數)
[] -> add(Src, Dst, B);
[_] -> fun () -> ok end
end;
{error, _} = Err ->
rabbit_misc:const(Err)
end;
{error, _} = Err ->
rabbit_misc:const(Err)
end
end, fun not_found_or_absent_errs/1).
在對應的exchange類型中校驗其合法性,校驗正常後,需要讀取mnesia數據庫中rabbit_route表中數據,查看是否有相應的binding信息,如果有則不再進行binding,沒有則進行binding操作,判斷rabbit_route表數據的原因是:不管exchange和queue是否需要持久化,最後都會將其binding信息寫入到rabbit_route表中。
這裏提一下,RabbitMQ所保存的一些metadata信息依賴於mnesia分佈式數據庫,其中RabbitMQ所使用的表可以通過以下命令進行顯示:
[root@master scripts]# ./rabbitmqctl eval 'mnesia:info().'
……
===> System info in version "4.16", debug level = none <===
opt_disc. Directory "/var/lib/rabbitmq/mnesia/rabbit@master" is used.
use fallback at restart = false
running db nodes = [rabbit@master]
stopped db nodes = []
master node tables = []
remote = []
ram_copies = [gm_group,mirrored_sup_childspec,rabbit_exchange,
rabbit_exchange_serial,
rabbit_exchange_type_consistent_hash,
rabbit_exchange_type_consistent_hash_ring_state,
rabbit_listener,rabbit_queue,rabbit_reverse_route,
rabbit_route,rabbit_semi_durable_route,
rabbit_topic_trie_binding,rabbit_topic_trie_edge,
rabbit_topic_trie_node,rh_exchange_table,
tracked_connection_on_node_rabbit@master,
tracked_connection_per_vhost_on_node_rabbit@master,
x_jms_topic_table]
disc_copies = [rabbit_durable_exchange,rabbit_durable_queue,
rabbit_durable_route,rabbit_runtime_parameters,
rabbit_topic_permission,rabbit_user,
rabbit_user_permission,rabbit_vhost,schema]
disc_only_copies = []
[{rabbit@master,disc_copies}] = [rabbit_runtime_parameters,
rabbit_durable_exchange,rabbit_durable_queue,
rabbit_user,rabbit_durable_route,
rabbit_topic_permission,rabbit_vhost,schema,
rabbit_user_permission]
[{rabbit@master,ram_copies}] = [rabbit_topic_trie_node,
tracked_connection_per_vhost_on_node_rabbit@master,
x_jms_topic_table,
rabbit_exchange_type_consistent_hash_ring_state,
rabbit_reverse_route,
rabbit_topic_trie_binding,
rabbit_exchange_type_consistent_hash,gm_group,
rabbit_listener,mirrored_sup_childspec,
rabbit_exchange,rabbit_route,
rabbit_exchange_serial,
tracked_connection_on_node_rabbit@master,
rabbit_semi_durable_route,rh_exchange_table,
rabbit_queue,rabbit_topic_trie_edge]
21 transactions committed, 7 aborted, 0 restarted, 0 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
ok
其中disc_copies表示會將數據存入磁盤和內存,ram_copies表示將數據存入內存。disc_only_copies表示將數據只存入磁盤。由於mnesia底層是基於ets和dets實現,所以可以通過ets或者dets查看錶中數據,注意ets和dets是隻能查詢本節點的表數據信息。如查看rabbit_exchang表數據信息:
[root@master scripts]# ./rabbitmqctl eval 'ets:tab2list(rabbit_exchange).'
[{exchange,{resource,<<"/">>,exchange,<<"amq.direct">>},
direct,true,false,false,[],undefined,undefined,undefined,
{[],[rabbit_event_exchange_decorator]},
#{}},
……
{exchange,{resource,<<"/">>,exchange,<<"amq.headers">>},
headers,true,false,false,[],undefined,undefined,undefined,
{[],[rabbit_event_exchange_decorator]},
#{}}]
具體參看erlang官方文檔的mnesia,ets和dets的使用說明。繼續回到binding操作,假設第一次進行binding操作,則將執行下列操作:
%% rabbit_binding.erl
%% 增加綁定的實際操作(實際操作mnesia數據庫表的函數)
add(Src, Dst, B) ->
[SrcDurable, DstDurable] = [durable(E) || E <- [Src, Dst]],
case (SrcDurable andalso DstDurable andalso
mnesia:read({rabbit_durable_route, B}) =/= []) of
false -> %% 根據交換機exchange和隊列的持久化狀態,將路由信息寫入mnesia數據庫表
ok = sync_route(#route{binding = B}, SrcDurable, DstDurable,
fun mnesia:write/3),
%% 交換機exchange創建後的回調
x_callback(transaction, Src, add_binding, B),
%% 讓Src對應的exchange交換機類型對應的模塊和所有的修飾模塊回調serialise_events函數,如果執行成功,則將XName在rabbit_exchange_serial表中的next值加一
Serial = rabbit_exchange:serial(Src),
fun () ->
x_callback(Serial, Src, add_binding, B),
%% 向rabbit_event事件中心發佈綁定信息被創建的事件
ok = rabbit_event:notify(binding_created, info(B))
end;
true -> rabbit_misc:const({error, binding_not_found})
end.
sync_route函數將exchange和queue的binding信息根據exchange和queue是否持久化來判斷是否寫入rabbit_durable_route和rabbit_semi_durable_route表中,但無論是否持久化都會寫入rabbit_route和rabbit_reverse_route表中。即
- rabbit_durable_route表:exchange和queue都持久化則寫入該表。
- rabbit_semi_durable_route表:exchange不需要持久化,queue需要持久化則寫入該表。
- rabbit_route和rabbit_reverse_route表:exchange和queue是否持久化都會寫入該表。
除了binding信息寫入上述表以外,對於類型爲topic的exchange,RabbitMQ還會將binding信息寫入rabbit_topic_trie_node,rabbit_topic_trie_edge和rabbit_topic_trie_binding表中。
%% rabbit_exchange_type_topic.erl
%% 路由綁定信息的添加回調該模塊進行相關的處理(transaction:事務)
add_binding(transaction, _Exchange, Binding) ->
internal_add_binding(Binding);
%% rabbit_exchange_type_topic.erl
%% exchange交換機內部添加綁定信息的接口
internal_add_binding(#binding{source = X, key = K, destination = D,
args = Args}) ->
%% 創建新的節點和邊
FinalNode = follow_down_create(X, split_topic_key(K)),
%% 創建綁定信息
trie_add_binding(X, FinalNode, D, Args),
ok.
其內部原理可參考以下鏈接:https://www.erlang-solutions.com/blog/rabbit-s-anatomy-understanding-topic-exchanges.html
再提一下,我們執行rabbitmqctl list_bindings命令時,查看到的exchange和queue的bindings信息是從rabbit_route表中進行獲取的。即
%% rabbit_binding.erl
%% 列出VHostPath下的所有路由綁定信息
list(VHostPath) ->
VHostResource = rabbit_misc:r(VHostPath, '_'),
Route = #route{binding = #binding{source = VHostResource,
destination = VHostResource,
_ = '_'},
_ = '_'},
[B || #route{binding = B} <- mnesia:dirty_match_object(rabbit_route,
Route)].
2. 總結
1 RabbitMQ的exchange和queue的binding的建立在客戶端與RabbitMQ之間建立連接且exchange和queue創建完成後。
2 binding信息會被保存到mnesia數據庫中,其保存的表爲rabbit_route表,根據exchange和queue是否持久化判斷是否保存到rabbit_durable_route和rabbit_semi_durable_route表中。
3 對於topic類型的exchange,其binding信息還會被保存到rabbit_topic_trie_node,rabbit_topic_trie_edge和rabbit_topic_trie_binding表中。