介紹
rabbitmq是基于Erlang語言編寫的一種消息隊列中間件,具體的內容網上有很多這里就不贅述了,本文主要介紹一下在python當中基于第三方庫pika
對rabbitmq的簡單使用
安裝
服務端
ubuntu安裝參考
https://www.cnblogs.com/vipstone/p/9184314.html
centos參考
https://blog.csdn.net/zhuzhezhuzhe1/article/details/80464291
客戶端
pip install pika
場景
- 任務隊列
- 發布訂閱內容
- ...
生產者-消費者模式
在消息隊列當中,最簡單的就是生產者消費者模式,即生產者發布一條消息,一個或者多個消費者在監聽等待,最終發布的消息被其中一個消費者給取走執行的模式,下面是簡單的示例:
生產者
import pika
credentials = pika.PlainCredentials('test', 'test')
# rabbitmq隊列賬號密碼
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1', credentials=credentials))
# 連接配置:主要包括host、port(默認15672)、credentials(登錄用戶名密碼)
channel = connection.channel()
# 創建一個連接通道
channel.queue_declare(queue='aaa')
# 聲明一個名為aaa的隊列
channel.basic_publish(exchange='',
routing_key='aaa',
body='this is a msg!')
# 往aaa隊列發送一條消息
connection.close()
#關閉連接
消費者
import pika
credentials = pika.PlainCredentials('test', 'test')
# rabbitmq隊列賬號密碼
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1', credentials=credentials))
# 連接配置:主要包括host、port(默認5672)、credentials(登錄用戶名密碼)
channel = connection.channel()
# 創建一個連接通道
channel.queue_declare(queue='aaa')
# 聲明一個名為aaa的隊列
def callback(ch, method, properties, body):
print("Received msg: {}".format(body))
channel.basic_consume('aaa', callback, auto_ack=True)
# 從aaa隊列中取出一條消息,并執行對應回調,第三個參數代表取到消息后自動回復執行完成,生產者會將該任務消息刪除
# 在舊版里傳參順序和參數名可能有所不同(舊版里是:callback, queue='aaa', no_ack=True)
print('start...')
channel.start_consuming()
# 開啟循環監聽消息隊列
此時如果打開多個消費者,那么可以發現生產者的發送的消息隊列將會被消費者按順序取走
生產者消費者模式-常用配置
任務完成回復
auto_ack
參數可以配置任務是否需要回復,默認是False
,即任務被取走之后,只有消費者在回調當中執行了ch.basic_ack(delivery_tag=method.delivery_tag)
方法以后,代表任務執行完成,此時生產者才會把任務從消息隊列當中刪除,若消費者沒能在關閉前執行上面那句方法,那么別的消費者將會在之后取走該任務去執行,直到生產者接收到執行完成的指令為止。如果auto_ack
值為True
,那么當任務被取走之后,生產者將直接把隊列中的任務刪除。舉例:
# 消費者
import pika
credentials = pika.PlainCredentials('test', 'test')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1', credentials=credentials))
channel = connection.channel()
channel.queue_declare(queue='aaa')
def callback(ch, method, properties, body):
print("Received msg: {}".format(body))
ch.basic_ack(delivery_tag=method.delivery_tag)
# 回復任務完成
channel.basic_consume('aaa', callback, auto_ack=False)
# 設置auto_ack=False,此時消費者必須提供任務完成的回復
print('start...')
channel.start_consuming()
消息隊列持久化
在隊列聲明時,可以通過參數durable
配置是否需要持久化,默認為False
,即不需要持久化,此時如果服務端掛了,那么消息隊列的內容將會丟失。如果配置持久化,那么首先需要在聲明當中設置durable=True
,然后在發布時也配置分發模式為持久化分發,舉例:
# 生產者
import pika
credentials = pika.PlainCredentials('test', 'test')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1', credentials=credentials))
channel = connection.channel()
channel.queue_declare(queue='bbb', durable=True)
# 聲明一個消息持久化的隊列,因為之前已經創建了aaa隊列,并且不是持久化的隊列,所以這里新建一個不存在的隊列
channel.basic_publish(exchange='',
routing_key='bbb',
body="this is a durable msg!",
properties=pika.BasicProperties(
delivery_mode=2,
# 配置消息持久化
))
connection.close()
閑置消費
默認是按照客戶端的順序一個個循環派發任務的,但要是第一個客戶端沒執行完,而下一個客戶端已經執行完了,此時如果還把任務派發給第一個就有些不好了,所以需要將任務派發給閑置的客戶端,類似于Nginx的負載均衡,實現只需要在消費者當中加入一句代碼:channel.basic_qos(prefetch_count=1)
,舉例:
# 消費者
import pika
credentials = pika.PlainCredentials('test', 'test')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1', credentials=credentials))
channel = connection.channel()
channel.queue_declare(queue='aaa')
def callback(ch, method, properties, body):
print("Received msg: {}".format(body))
ch.basic_ack(delivery_tag=method.delivery_tag)
channel.basic_qos(prefetch_count=1)
# 設置閑置消費
channel.basic_consume('aaa', callback, auto_ack=False)
print('start...')
channel.start_consuming()
發布訂閱模式-fanout
前面介紹的都是生產者發布一條任務消息,然后一個或者多個消費者中的其中一個取走這個任務去執行的情況。而有一種場景,如廣播、微信公眾號的消息推送這種,往往需要將一條消息發布給所有的消費者執行,而在rabbitmq當中就可以通過創建一個exchange
交換器來創建和管理多個隊列,即一個exchange
下有多個隊列,并且每個隊列對應一個消費者,當有消息的時候,exchange
會將消息發送給自己所管理的所有隊列,此時需要設置類型為fanout
,舉例:
發布者
import pika
credentials = pika.PlainCredentials('test', 'test')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1', credentials=credentials))
channel = connection.channel()
channel.exchange_declare(exchange='ex1', exchange_type='fanout')
# 聲明一個名為ex1的交換器,發布類型為集體分發
channel.basic_publish(exchange='ex1',
routing_key='',
body="everyone on ex1 will get this msg!"
)
# 往ex1中所有隊列分發消息
connection.close()
訂閱者
import pika
credentials = pika.PlainCredentials('test', 'test')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1', credentials=credentials))
channel = connection.channel()
channel.exchange_declare(exchange='ex1', exchange_type='fanout')
# 聲明一個名為ex1的交換器,發布類型為集體分發
result = channel.queue_declare(queue='', exclusive=True)
# 這里不定義名字,通過exclusive=True生成一個名字不重復的隊列
queue_name = result.method.queue
# 獲取隊列名
channel.queue_bind(queue_name, exchange='ex1')
# 綁定隊列名和ex1交換器
def callback(ch, method, properties, body):
print("Received msg: {}".format(body))
channel.basic_consume(queue_name, callback, True)
# 使用隨機生成的隊列名在ex1交換器下接收消息
print('queue:{} start...'.format(queue_name))
channel.start_consuming()
注:
發布訂閱模式和生產消費者模式還有一點不同就是:生產消費者模式當中,生產者產生的消費只要沒被取走,那么消息就會一直留著等待被消費者取走;而在發布訂閱模式當中,發布者只會發布一次,發布完該消息就會被刪除,因此如果訂閱者沒有在發布者發布時接收到消息,將永遠錯過接收的機會(就像關注公眾號以后,公眾號并不會把以前的所有歷史推送信息也給你重新再推送一遍一樣)
指定發布訂閱-direct
對于發布時,可以不給所有隊列發送消息,而是指定給哪些隊列發送,舉例:
發布者
import pika
credentials = pika.PlainCredentials('test', 'test')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1', credentials=credentials))
channel = connection.channel()
channel.exchange_declare(exchange='ex2', exchange_type='direct')
# 聲明一個名為ex2的交換器,發布類型為指定路由分發
channel.basic_publish(exchange='ex2',
routing_key='test1',
body="only test1 will get this msg!"
)
# 往ex2中test1的路由分發消息
channel.basic_publish(exchange='ex2',
routing_key='test2',
body="only test2 will get this msg!"
)
# 往ex2中test2的路由分發消息
connection.close()
訂閱者
import pika
credentials = pika.PlainCredentials('test', 'test')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1', credentials=credentials))
channel = connection.channel()
channel.exchange_declare(exchange='ex2', exchange_type='direct')
# 聲明一個名為ex2的交換器,接收類型為指定路由
result = channel.queue_declare(queue='', exclusive=True)
queue_name = result.method.queue
channel.queue_bind(queue_name, exchange='ex2', routing_key='test1')
# 綁定ex2與隊列,以及綁定對應的路由
channel.queue_bind(queue_name, exchange='ex2', routing_key='test2')
channel.queue_bind(queue_name, exchange='ex2', routing_key='test3')
# 可以一個隊列綁定多個路由
def callback(ch, method, properties, body):
print("Received msg: {}".format(body))
channel.basic_consume(queue_name, callback, True)
print('queue:{} start...'.format(queue_name))
channel.start_consuming()
模糊匹配發布訂閱-topic
在發布訂閱時也可以通過模糊匹配路由,當符合匹配規則的路由將會接收到消息,其中常用的通配符有*
和#
,*
后面能夠匹配一個單詞,#
后面能夠匹配一個或多個單詞(例如有a.#
和a.*
的匹配規則,那么a.x
能被兩個規則都匹配到,但a.x.y
只能被a.#
匹配到),舉例:
發布者
import pika
credentials = pika.PlainCredentials('test', 'test')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1', credentials=credentials))
channel = connection.channel()
channel.exchange_declare(exchange='ex3', exchange_type='topic')
# 聲明一個名為ex3的交換器,發布類型為指定模糊匹配路由分發
channel.basic_publish(exchange='ex3',
routing_key='test.a',
body="test.# or test.* will get this msg!"
)
channel.basic_publish(exchange='ex3',
routing_key='test.b.c',
body="only test.# will get this msg!"
)
connection.close()
訂閱者
import pika
credentials = pika.PlainCredentials('test', 'test')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1', credentials=credentials))
channel = connection.channel()
channel.exchange_declare(exchange='ex3', exchange_type='topic')
# 聲明一個名為ex3的交換器,發布類型為模糊匹配路由接收
result = channel.queue_declare(queue='', exclusive=True)
queue_name = result.method.queue
channel.queue_bind(queue_name, exchange='ex3', routing_key='test.#')
# 這里因為routing_key='test.#',所以test.a和test.b.c的消息都能接收到
def callback(ch, method, properties, body):
print("Received msg: {}".format(body))
channel.basic_consume(queue_name, callback, True)
print('queue:{} start...'.format(queue_name))
channel.start_consuming()
RPC
遠程過程調用,一個簡單的理解就是假如本地要調用一個函數,而服務端已經實現了該函數,那么可以向服務端請求調用該函數,并把返回值返回給本地,具體參考:https://www.cnblogs.com/goldsunshine/p/8665456.html
其他操作
在pika
模塊當中也提供了如刪除/解綁隊列、刪除/解綁交換器等操作,舉例:
import pika
credentials = pika.PlainCredentials('test', 'test')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1', credentials=credentials))
channel = connection.channel()
channel.exchange_delete("ex1")
# 刪除交換器
channel.queue_delete("aaa")
# 刪除隊列
connection.close()
更多關于pika操作參考:https://www.cnblogs.com/cwp-bg/p/8426188.html