參考來源:
https://www.cnblogs.com/alex3714/p/6351797.html
http://www.cnblogs.com/dachenzi/p/8082730.html
https://www.cnblogs.com/alex3714/p/6351797.html
本文主要是做記錄,便于自己往后用到的時候能找到相關(guān)的資料
第1步在:pycham 中 創(chuàng)建項目名稱 proj(創(chuàng)建python 包)
第2步在:建立相關(guān)的python項目文件
編寫:celery.py文件內(nèi)容
"""
@Create_Time: 2018/1/9 17:33
@File: celery.py
@文件功能描述:
"""
from __future__ import absolute_import, unicode_literals
from celery import Celery
broker = 'redis://:123456@localhost:6379/0'
backend = 'redis://:123456@localhost:6379/1'
app = Celery('proj',
broker=broker,
backend=backend,
include=['proj.tasks'])
# Optional configuration, see the application user guide.
app.conf.update(
result_expires=3600,
)
if __name__ == '__main__':
app.start()
編寫:tasks.py文件內(nèi)容
#!/usr/bin/evn python
# coding=utf-8
from .celery import app # 導入應用
@app.task
def add(x, y):
return x + y
@app.task
def minus(x, y):
return x - y
查看相關(guān)的項目目錄情況:
# yum -y install tree
[root@localhost proj]# tree
.
├── celery.py
├── __init__.py
└── tasks.py
0 directories, 3 files
第3步在:后臺啟動多進程celery work
上一篇筆記中使用的方式是前臺啟動的方式,若終端結(jié)束,則相關(guān)的進程也會結(jié)束,并且有相關(guān)的日志輸出,實際工作中是一般不允許的!
PS: 后臺啟動worker
$ celery multi start w1 -A proj -l info
celery multi v4.0.0 (latentcall)
Starting nodes...
w1.halcyon.local: OK
補充命令信息:
啟動:
$ celery multi start w1 -A proj -l info
重啟:
$ celery multi restart w1 -A proj -l info
立即停止
$ celery multi stop w1 -A proj -l info
延遲停止,確保了所有正在執(zhí)行的任務是完成之前退出:
$ celery multi stopwait w1 -A proj -l info
首先進入到當前的項目的同級目錄下:
# 需在 proj的同一級目錄(在同級目錄下執(zhí)行:就是在項目名稱同一個目錄中?。。?執(zhí)行celery 后臺啟動 celery worker進程
celery multi start work_1 -A proj
或
celery multi start work_1 -A /data/app/proj
# work_1 為woker的名稱,可以用來進行對該進程進行管理
后臺運行成功
導入任務模塊進行使用:
>>> import appcelery
>>> tasks.add.delay(3,676)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'tasks' is not defined
>>> from appcelery import tasks
>>> tasks.add.delay(3,676)
<AsyncResult: b9d200ed-8b87-49c3-8ff2-744b58a4cafa>
>>>
查看對應的redis數(shù)據(jù)信息:
image.png
執(zhí)行3次:>>> tasks.add.delay(3,676)
image.png
運行錯誤:
image.png
image.png
在同級目錄下執(zhí)行:就是在項目名稱同一個目錄中?。。?/p>
image.png
原因:配置文件的redis的寫錯!
image.png
補充:
如果你手動的清除了相關(guān)的信息 需要重新再啟動一下celery
就是需要重新再次執(zhí)行以下:
# celery -A proj2 worker -l info
然后測試導入的話是需要使用:
[root@localhost app]# python
Python 3.5.2 (default, Jan 8 2018, 09:14:33)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import proj2.tasks
>>> proj2.tasks.add.delay(222,344)
<AsyncResult: 6ac6626e-7fa5-40b7-b37f-064b6d448cc1>
>>> proj2.tasks.add.delay(222,3443453)
<AsyncResult: 81ef3703-422a-490f-8122-0ab7d8a631ad>
>>>
# 為了導入方便,這里在appcelery同級目錄下 執(zhí)行python解釋器
[root@namenode celerymodule]# ls
appcelery __pycache__ task.py
[root@namenode celerymodule]# python3
Python 3.6.4 (default, Dec 25 2017, 17:45:03)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from appcelery import tasks # 相對導入
>>> tasks.add.delay(1,2) # 執(zhí)行任務
<AsyncResult: bc90bfa6-7972-4679-b897-4f5a98c70ae0>
>>>
補充:celery的其他參數(shù)或命令
[root@localhost proj]# celery
usage: celery <command> [options]
Show help screen and exit.
positional arguments:
args
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
Global Options:
-A APP, --app APP
-b BROKER, --broker BROKER
--loader LOADER
--config CONFIG
--workdir WORKDIR
--no-color, -C
--quiet, -q
---- -- - - ---- Commands- -------------- --- ------------
+ Main:
| celery worker
| celery events
| celery beat
| celery shell
| celery multi
| celery amqp
+ Remote Control:
| celery status
| celery inspect --help
| celery inspect active
| celery inspect active_queues
| celery inspect clock
| celery inspect conf [include_defaults=False]
| celery inspect memdump [n_samples=10]
| celery inspect memsample
| celery inspect objgraph [object_type=Request] [num=200 [max_depth=10]]
| celery inspect ping
| celery inspect query_task [id1 [id2 [... [idN]]]]
| celery inspect registered [attr1 [attr2 [... [attrN]]]]
| celery inspect report
| celery inspect reserved
| celery inspect revoked
| celery inspect scheduled
| celery inspect stats
| celery control --help
| celery control add_consumer <queue> [exchange [type [routing_key]]]
| celery control autoscale [max [min]]
| celery control cancel_consumer <queue>
| celery control disable_events
| celery control election
| celery control enable_events
| celery control heartbeat
| celery control pool_grow [N=1]
| celery control pool_restart
| celery control pool_shrink [N=1]
| celery control rate_limit <task_name> <rate_limit (e.g., 5/s | 5/m | 5/h)>
| celery control revoke [id1 [id2 [... [idN]]]]
| celery control shutdown
| celery control terminate <signal> [id1 [id2 [... [idN]]]]
| celery control time_limit <task_name> <soft_secs> [hard_secs]
+ Utils:
| celery purge
| celery list
| celery call
| celery result
| celery migrate
| celery graph
| celery upgrade
+ Debugging:
| celery report
| celery logtool
---- -- - - --------- -- - -------------- --- ------------
Type 'celery <command> --help' for help using a specific command.
# 多進程相關(guān)
celery multi stop WOERNAME # 停止worker進程,有的時候這樣無法停止進程,就需要加上-A 項目名,才可以刪掉
celery multi restart WORKNAME # 重啟worker進程
# 查看進程數(shù)
celery status -A celery # 查看該項目運行的進程數(shù)
一些錯誤的解決:
示例在windows下運行的時候:
執(zhí)行:
D:\python_learn\celeryTest>celery -A tasks worker --loglever=info
因為proj才是模塊的名稱!
image.png
如果是:D:\python_learn\celeryTest>celery -A tasks worker --loglever=info
對應的模塊應該是:
# coding:utf-8
from celery import Celery
app = Celery('tasks', broker='amqp://guest@localhost//')
@app.task
def add(x, y):
return x + y
windos下執(zhí)行:
D:\python_learn\celeryTest>celery -A proj worker --loglever=info
usage: celery worker [options]
celery: error: unrecognized arguments: --loglever=info
錯誤原因估計最新版本不支持!是不支持windos!
卸載重新安裝最新版本:
D:\python_learn\celeryTest>pip uninstall celery
Uninstalling celery-4.1.0:
重新安裝最新版本:
pip install celery==3.1.18