kombu升级后得到<>对于task_id,不是JSON可序列化的

时间:2017-05-10 07:19:28

标签: python json celery kombu

升级后我得到ERROR EncodeError(TypeError('6JQAKHNMG9 is not JSON serializable',),)

成功安装:

  • amqp-2.1.4
  • billiard-3.5.0.2
  • 芹菜-4.0.2
  • kombu-4.0.2
  • pytz-2017.2

(最初需要Flower 0.9.1安装)

正在运行:

  • AMQP == 1.4.9
  • 台球== 3.3.0.23
  • 芹菜== 3.1.23
  • 海带== 3.0.35
  • pytz == 2016.10

现在我明白了:

05/10/2017 06:37:55.045 ERROR EncodeError(TypeError('6JQAKHNMG9 is not JSON serializable',),)
Traceback (most recent call last):
  File "/usr/local/src/gonzo/api/version1_0/application/api_main.py", line 191, in post
    retries=3)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/base.py", line 737, in send_task
    amqp.send_task_message(P, name, message, **options)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/amqp.py", line 558, in send_task_message
    **properties
  File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 169, in publish
    compression, headers)
  File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 252, in _prepare
    body) = dumps(body, serializer=serializer)
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 221, in dumps
    payload = encoder(data)
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 54, in _reraise_errors
    reraise(wrapper, wrapper(exc), sys.exc_info()[2])
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 50, in _reraise_errors
    yield
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 221, in dumps
    payload = encoder(data)
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/json.py", line 72, in dumps
    **dict(default_kwargs, **kwargs))
  File "/usr/lib/python2.7/json/__init__.py", line 251, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/json.py", line 62, in default
    return super(JSONEncoder, self).default(o)
  File "/usr/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
EncodeError: 6JQAKHNMG9 is not JSON serializable

我的代码:

我传递了一个自定义的“task_id”,它是一个字符串,(不是JSON对象)它在升级之前用来工作(如果我还原版本再次工作),传递自定义task_id的正确格式是什么?

celery.send_task('process_campaign',
                             exchange='gonzo',
                             queue='gold',
                             routing_key='gonzo.gold',
                             kwargs={'campaign_instance': campaign_instance},
                             task_id=campaign_instance.reference,
                             retries=3)

我试过了:

task_id=json.dumps(campaign_instance.reference) or
task_id={'task_id': campaign_instance.reference}

同样的错误,任何想法,我找不到这个问题的文档?

1 个答案:

答案 0 :(得分:3)

version 4.0起,celery中的默认序列化程序为'json'。在默认值为'pickle'之前。 (我不知道kombu包版本的详细信息)

发送消息时,您传递的campaign_instance参数可能不是JSON可序列化的,但是PICKLE可序列化,这就是它之前工作的原因。

您可以在指定'pickle'的设置中更改默认序列化程序,这是最短的方式:http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-task_serializer

或者您可以修改代码以仅将JSON可序列化参数传递给您的任务。