Remote procedure call（RPC）
圖解：
我們的RPC將會這樣執行：
> 當客戶端啓動後，它創建一個隨機的回調隊列
> 對一個RPC請求，客戶端發送一個消息包含兩個屬性：reply_to（用來設置回調隊列）和 correlation_id（用來爲每個請求設置一個唯一標識）
> 請求發送到RPC_Queue隊列
> Server服務端監聽RPC_Queue隊列等待請求，當請求出現，服務端獲取信息並將處理結果返回給客戶端，使用reply_to字段中的隊列
> 客戶端在callback隊列中等待數據，當一個消息出現後，檢查這個correlation_id屬性,如果和請求中的值匹配將返回給應用
Client端RPC_Client模塊： import pika class Client(object): def __init__(self, sever_list): self.sever_list = sever_list self.connection = pika.BlockingConnection(pika.ConnectionParameters(host='172.16.111.134')) self.channel = self.connection.channel() self.channel.exchange_declare(exchange='rpc', type='direct') result = self.channel.queue_declare(exclusive=True) self.callback_queue = result.method.queue self.channel.basic_consume(self.on_response, no_ack=True, queue=self.callback_queue) # 綁定用來接收服務端消息的隊列並監聽 def on_response(self, ch, method, props, body): # 接收服務端信息 for corr_id in self.sever_list: if corr_id == props.correlation_id: self.response.append(body.decode()) def call(self, cmd): # 發送消息的函數 self.response = [] # self.corr_id = str(uuid.uuid4()) for corr_id in self.sever_list: # 發佈者，將corr_id既當routing_key，又當correlation_id傳遞 self.channel.basic_publish(exchange='rpc', routing_key=corr_id, properties=pika.BasicProperties(reply_to=self.callback_queue, correlation_id=corr_id,), body=cmd) while not self.response or len(self.response) != len(self.sever_list): # 一個小bug就是如果長度不等，就會一直hang on .. self.connection.process_data_events() return self.response Server端import pika
import subprocess
get_server_ip = '/sbin/ifconfig|grep "inet addr"|grep -v 127.0.0.1| \
sed -e "s/^.*addr://;s/Bcast.*$//"'
result = subprocess.Popen(get_server_ip, shell=True, stdout=subprocess.PIPE)
ip = result.stdout.read().decode().strip() # 獲取本機IP
connection = pika.BlockingConnection(pika.ConnectionParameters(host='172.16.111.134'))
channel = connection.channel()
channel.exchange_declare(exchange='rpc', type='direct')
result = channel.queue_declare(exclusive=True)
queue_name = result.method.queue
channel.queue_bind(exchange='rpc', queue=queue_name, routing_key=ip)
def task_mission(cmd):
result = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
result = result.stdout.read()
if not result:
return ip + ': 輸入命令不合法\n'
return ip + ':\n' + result.decode()
def on_request(ch, method, props, body): # 接收客戶端信息
body = body.decode()
print(" [.] 執行的命令：%s" % body)
response = task_mission(body) # 處理消息
ch.basic_publish(exchange='', # 再將返回值發送給客戶端
routing_key=props.reply_to,
properties=pika.BasicProperties(correlation_id=props.correlation_id),
body=response)
ch.basic_ack(delivery_tag=method.delivery_tag)
channel.basic_qos(prefetch_count=1)
channel.basic_consume(on_request, queue=queue_name)
print(" [x] Awaiting RPC requests")
channel.start_consuming() |
###########################################################################################
以下摘自RabbitMQ官網內容：
Client interface
To illustrate how an RPC service could be used we're going to create a simple client class. It's going to expose a method named call which sends an RPC request and blocks until the answer is received:
fibonacci_rpc = FibonacciRpcClient()
result = fibonacci_rpc.call(4)
print("fib(4) is %r" % result)
Callback queue
In general doing RPC over RabbitMQ is easy. A client sends a request message and a server replies with a response message. In order to receive a response the client needs to send a 'callback' queue address with the request. Let's try it:
result = channel.queue_declare(exclusive=True) callback_queue = result.method.queue channel.basic_publish(exchange='', routing_key='rpc_queue', properties=pika.BasicProperties( reply_to = callback_queue, ), body=request) # ... and some code to read a response message from the callback_queue ...
Correlation id
In the method presented above we suggest creating a callback queue for every RPC request. That's pretty inefficient, but fortunately there is a better way - let's create a single callback queue per client.
That raises a new issue, having received a response in that queue it's not clear to which request the response belongs. That's when the correlation_id property is used. We're going to set it to a unique value for every request. Later, when we receive a message in the callback queue we'll look at this property, and based on that we'll be able to match a response with a request. If we see an unknown correlation_id value, we may safely discard the message - it doesn't belong to our requests.
You may ask, why should we ignore unknown messages in the callback queue, rather than failing with an error? It's due to a possibility of a race condition on the server side. Although unlikely, it is possible that the RPC server will die just after sending us the answer, but before sending an acknowledgment message for the request. If that happens, the restarted RPC server will process the request again. That's why on the client we must handle the duplicate responses gracefully, and the RPC should ideally be idempotent.
Summary
Our RPC will work like this:
>When the Client starts up, it creates an anonymous exclusive callback queue.
>For an RPC request, the Client sends a message with two properties: reply_to, which is set to the callback queue and correlation_id, which is set to a unique value for every request.
>The request is sent to an rpc_queue queue.
>The RPC worker (aka: server) is waiting for requests on that queue. When a request appears, it does the job and sends a message with the result back to the Client, using the queue from the reply_to field.
>The client waits for data on the callback queue. When a message appears, it checks the correlation_id property. If it matches the value from the request it returns the response to the application.
Putting it all together
The code for rpc_server.py:
#!/usr/bin/env python
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
channel = connection.channel()
channel.queue_declare(queue='rpc_queue')
def fib(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fib(n-1) + fib(n-2)
def on_request(ch, method, props, body):
n = int(body)
print(" [.] fib(%s)" % n)
response = fib(n)
ch.basic_publish(exchange='',
routing_key=props.reply_to,
properties=pika.BasicProperties(correlation_id = \
props.correlation_id),
body=str(response))
ch.basic_ack(delivery_tag = method.delivery_tag)
channel.basic_qos(prefetch_count=1)
channel.basic_consume(on_request, queue='rpc_queue')
print(" [x] Awaiting RPC requests")
channel.start_consuming()The server code is rather straightforward:
>(4) As usual we start by establishing the connection and declaring the queue.
>(11) We declare our fibonacci function. It assumes only valid positive integer input. (Don't expect this one to work for big numbers, it's probably the slowest recursive implementation possible).
>(19) We declare a callback for basic_consume, the core of the RPC server. It's executed when the request is received. It does the work and sends the response back.
>(32) We might want to run more than one server process. In order to spread the load equally over multiple servers we need to set the prefetch_count setting.
The code for rpc_client.py:
#!/usr/bin/env python
import pika
import uuid
class FibonacciRpcClient(object):
def __init__(self):
self.connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
self.channel = self.connection.channel()
result = self.channel.queue_declare(exclusive=True)
self.callback_queue = result.method.queue
self.channel.basic_consume(self.on_response, no_ack=True,
queue=self.callback_queue)
def on_response(self, ch, method, props, body):
if self.corr_id == props.correlation_id:
self.response = body
def call(self, n):
self.response = None
self.corr_id = str(uuid.uuid4())
self.channel.basic_publish(exchange='',
routing_key='rpc_queue',
properties=pika.BasicProperties(
reply_to = self.callback_queue,
correlation_id = self.corr_id,
),
body=str(n))
while self.response is None:
self.connection.process_data_events()
return int(self.response)
fibonacci_rpc = FibonacciRpcClient()
print(" [x] Requesting fib(30)")
response = fibonacci_rpc.call(30)
print(" [.] Got %r" % response)The client code is slightly more involved:
>(7) We establish a connection, channel and declare an exclusive 'callback' queue for replies.
>(16) We subscribe to the 'callback' queue, so that we can receive RPC responses.
>(18) The 'on_response' callback executed on every response is doing a very simple job, for every response message it checks if the correlation_id is the one we're looking for. If so, it saves the response in self.response and breaks the consuming loop.
>(23) Next, we define our main call method - it does the actual RPC request.
>(24) In this method, first we generate a unique correlation_id number and save it - the 'on_response' callback function will use this value to catch the appropriate response.
>(25) Next, we publish the request message, with two properties: reply_to and correlation_id.
>(32) At this point we can sit back and wait until the proper response arrives.
>(33) And finally we return the response back to the user.
Our RPC service is now ready. We can start the server:
$ python rpc_server.py [x] Awaiting RPC requests
To request a fibonacci number run the client:
$ python rpc_client.py [x] Requesting fib(30)
The presented design is not the only possible implementation of a RPC service, but it has some important advantages:
>If the RPC server is too slow, you can scale up by just running another one. Try running a second rpc_server.py in a new console.
>On the client side, the RPC requires sending and receiving only one message. No synchronous calls like queue_declare are required. As a result the RPC client needs only one network round trip for a single RPC request.
Our code is still pretty simplistic and doesn't try to solve more complex (but important) problems, like:
>How should the client react if there are no servers running?
>Should a client have some kind of timeout for the RPC?
>If the server malfunctions and raises an exception, should it be forwarded to the client?
>Protecting against invalid incoming messages (eg checking bounds) before processing.
鏈接地址（官網）：http://www.rabbitmq.com/tutorials/tutorial-six-python.html
鏈接地址（翻譯）：http://blog.csdn.net/songfreeman/article/details/50951065