Django中如何使用django-celery完成异步任务

安装Celery

我们可以使用pip在vietualenv中安装:

 pip install django-celery  celery

django settings设置

import djcelery

djcelery.setup_loader()

BROKER_URL = 'redis://127.0.0.1:6379/2' ---使用redis当消息队列

注册

INSTALLED_APPS = [

    'django.contrib.admin',

    'django.contrib.auth',

    'django.contrib.contenttypes',

    'django.contrib.sessions',

    'django.contrib.messages',

    'django.contrib.staticfiles',

    'devops',

    'apps',

    'common',

    'djcelery',

]

在apps下创建一个task.py文件

from celery import task

@task

def add(x, y):

    return x + y

@task

def pp():

return 'ffffffffffffffffffffff'

返回settings将其导入进去

import djcelery

djcelery.setup_loader()

BROKER_URL = 'redis://127.0.0.1:6379/2'

CELERY_IMPORTS = ('apps.task')

随意选个views中的方法调用改task测试下

#项目列表

def project_list(request):

    admin = Admin.objects.get(id=get_current_admin_id(request))

    pt=admin.projects.all().order_by('-id')

    from apps import task

    tt=task.add.delay(2,2)

    print 'vvvvvvvvvvvvvvvvvvvv',tt -----访问该功能时调用task的add方法

    page_objects = pages(pt, request, 5) ##分页

    return render_to_response('project/project_list.html',locals())

安装启动redis：略

启动runserver 与 celery

python manage.py runserver

python manage.py celery worker --loglevel=info

访问对应的页面看日志

[2017-11-10 14:55:17,016: INFO/MainProcess] Task apps.task.add[9bafe6d2-8411-4f5f-8eed-10444da0ae3a] succeeded in 0.00280212797225s: 4 --可以看到worker日志，返回结果

手工测试task

打开新的terminal, 激活virtualenv, 并切换到django项目目录:

    $ python manage.py shell

    >>> from apps.task import add

    >>> add.delay(2, 2)

此时, 你可以在worker窗口中看到worker执行该task:

    [2014-10-07 08:47:08,076: INFO/MainProcess] Got task from broker: myapp.tasks.add[e080e047-b2a2-43a7-af74-d7d9d98b02fc]

    [2014-10-07 08:47:08,299: INFO/MainProcess] Task myapp.tasks.add[e080e047-b2a2-43a7-af74-d7d9d98b02fc] succeeded in 0.183349132538s: 4

Eager模式

Eager模式

如果在settings.py设置:

    CELERY_ALWAYS_EAGER = True

那么Celery便以eager模式运行, 则task便不需要加delay运行:

    # 若启用eager模式, 则以下两行代码相同

    add.delay(2, 2)

    add(2, 2)

dj-celery 定时任务

settings配置添加

CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler' # 定时任务

CELERYBEAT_SCHEDULE = {

   'add-every-3-seconds': {

       'task': 'apps.task.pp',

       'schedule': timedelta(seconds=3) ---每隔3秒执行task下的pp函数

   },

}

如果需要传参可以这样写

CELERYBEAT_SCHEDULE = {

   'add-every-3-minutes': {

       'task': 'apps.task.add',

       'schedule': timedelta(seconds=3),

        'args': (16, 16)

   },

}

启动celery beat

启动beat

执行定时任务时, Celery会通过celerybeat进程来完成. Celerybeat会保持运行, 一旦到了某一定时任务需要执行时, Celerybeat便将其加入到queue中. 不像worker进程, Celerybeat只有需要一个即可.

启动：

python manage.py celery beat --loglevel=info

其实还有一种简单的启动方式worker和beat一起启动：

python manage.py celery worker --loglevel=info --beat

查看worker日志

[2017-11-10 16:17:25,853: INFO/MainProcess] Received task: apps.task.pp[8a3af6fb-5189-4647-91f2-8aa07489dd1e]

[2017-11-10 16:17:25,858: INFO/MainProcess] Task apps.task.pp[8a3af6fb-5189-4647-91f2-8aa07489dd1e] succeeded in 0.00379144400358s: 'ffffffffffffffffffffff'

[2017-11-10 16:17:28,858: INFO/MainProcess] Received task: apps.task.pp[d87e4ea0-8881-449a-b993-e7657f50ef25]

[2017-11-10 16:17:28,864: INFO/MainProcess] Task apps.task.pp[d87e4ea0-8881-449a-b993-e7657f50ef25] succeeded in 0.0049942266196s: 'ffffffffffffffffffffff'

[2017-11-10 16:17:31,859: INFO/MainProcess] Received task: apps.task.pp[4d05b4f3-92ff-4922-a8f4-7e047749239a]

[2017-11-10 16:17:31,865: INFO/MainProcess] Task apps.task.pp[4d05b4f3-92ff-4922-a8f4-7e047749239a] succeeded in 0.00537821277976s: 'ffffffffffffffffffffff'

[2017-11-10 16:17:34,859: INFO/MainProcess] Received task: apps.task.pp[5b21afc1-ebf1-4858-be68-20b9bf318452]

[2017-11-10 16:17:34,865: INFO/MainProcess] Task apps.task.pp[5b21afc1-ebf1-4858-be68-20b9bf318452] succeeded in 0.00530335493386s: 'ffffffffffffffffffffff'

参考文档：

http://www.codeweblog.com/djcelery%E5%85%A5%E9%97%A8-%E5%AE%9E%E7%8E%B0%E8%BF%90%E8%A1%8C%E5%AE%9A%E6%97%B6%E4%BB%BB%E5%8A%A1/

django-celery参考文档：
http://blog.csdn.net/acm_zl/article/details/53188064
https://www.cnblogs.com/znicy/p/5626040.html
http://www.weiguda.com/blog/73/
https://www.cnblogs.com/Lin-Yi/p/7590971.html

补充还有另外一种方法那就是celery

具体配置可见参考

http://www.jianshu.com/p/7a869a73b92f

ITPub博客

【Python】django-celery异步任务队列

Django中如何使用django-celery完成异步任务

安装Celery

最新文章