尝试运行python类静态方法时出现pickle错误

时间:2017-12-16 23:45:37

标签: python django pickle static-methods

我正在尝试在我的Django应用程序中的一个类上运行静态方法。我正在使用多处理模块使这个任务更快一些,因为该方法将迭代我的数据库中的大量对象。当我在本地运行它时它工作正常,但是当我在生产中运行它时,我得到了这个酸洗错误......

快速注意:我在本地使用python3.6,在生产中使用python3.4。这会影响我的物体的酸洗吗?

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/socialauto/social-auto-web/vehicle/models.py", line 193, in image_existence_check
    p.map(Vehicle.check_image, enumerate(vehicles))
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 599, in get
    raise self._value
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 383, in _handle_tasks
    put(task)
  File "/usr/lib/python3.4/multiprocessing/connection.py", line 206, in send
    self._send_bytes(ForkingPickler.dumps(obj))
  File "/usr/lib/python3.4/multiprocessing/reduction.py", line 50, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function Vehicle.check_image at 0x7f49974d5268>: attribute lookup check_image on vehicle.models failed

Por Que

模型方法:

    @staticmethod
    def check_image(veh):
        index = veh[0]
        print(index)

        veh = veh[1]

        try:
            images = veh.images.all()

            if images.count() == 1:
                image = images[0]

                response = requests.get(image.image_url)

                if response.status_code == 200:
                    veh.has_image = True
                else:
                    veh.has_image = False
            elif images.count() > 1:
                has_image = True

                for img in images:
                    response = requests.get(img.image_url)

                    if response != 200:
                        has_image = False

                veh.has_image = has_image

            else:

                veh.has_image = False

            veh.save()

        except Exception as e:
            logging.error(e)
            pass

    @staticmethod
    def image_existence_check():
        from time import time
        from multiprocessing.pool import Pool

        ts = time()
        vehicles = Vehicle.objects.all()[:100]

        # map(lambda (i, x): {'name': x, 'rank': i}, enumerate(ranked_users))

        with Pool(10) as p:
            print('Beginning')
            p.map(Vehicle.check_image, enumerate(vehicles))

        print('Took {}s'.format(time() - ts))

0 个答案:

没有答案