无法将对象放在Queue上

时间:2010-10-01 12:13:54

标签: python synchronization queue multiprocessing pickle

我想在scapy.layers.dhcp.BOOTP上放置multiprocessing.Queue的实例。每当我拨打put()时,都会发生以下异常:

Traceback (most recent call last):
  File "/usr/lib/python2.6/multiprocessing/queues.py", line 242, in _feed
    send(obj)
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

原因是尝试使用pickle.dumps()直接挑选实例也失败了。但为什么这个班级不可挑选?

对于所有没有安装scapy的人:

class BOOTP(Packet):
    name = "BOOTP"
    fields_desc = [ ByteEnumField("op",1, {1:"BOOTREQUEST", 2:"BOOTREPLY"}),
                    ByteField("htype",1),
                    ByteField("hlen",6),
                    ByteField("hops",0),
                    IntField("xid",0),
                    ShortField("secs",0),
                    FlagsField("flags", 0, 16, "???????????????B"),
                    IPField("ciaddr","0.0.0.0"),
                    IPField("yiaddr","0.0.0.0"),
                    IPField("siaddr","0.0.0.0"),
                    IPField("giaddr","0.0.0.0"),
                    Field("chaddr","", "16s"),
                    Field("sname","","64s"),
                    Field("file","","128s"),
                    StrField("options","") ]
    def guess_payload_class(self, payload):
        if self.options[:len(dhcpmagic)] == dhcpmagic:
            return DHCP
        else:
            return Packet.guess_payload_class(self, payload)
    def extract_padding(self,s):
        if self.options[:len(dhcpmagic)] == dhcpmagic:
            # set BOOTP options to DHCP magic cookie and make rest a payload of DHCP options
            payload = self.options[len(dhcpmagic):]
            self.options = self.options[:len(dhcpmagic)]
            return payload, None
        else:
            return "", None
    def hashret(self):
        return struct.pack("L", self.xid)
    def answers(self, other):
        if not isinstance(other, BOOTP):
            return 0
        return self.xid == other.xid

还有其他方法可以将此实例“传输”到另一个子流程吗?

3 个答案:

答案 0 :(得分:1)

嗯,问题是你不能腌制功能类型。这是你type(some_user_function)时得到的。见:

>>> import types
>>> pickle.dumps(types.FunctionType)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'picke' is not defined
>>> pickle.dumps(types.FunctionType)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python26\lib\pickle.py", line 1366, in dumps
    Pickler(file, protocol).dump(obj)
  File "C:\Python26\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "C:\Python26\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python26\lib\pickle.py", line 748, in save_global
    (obj, module, name))
pickle.PicklingError: Can't pickle <type 'function'>: it's not found as __built
n__.function

因此,这样的函数类型存储在您尝试发送的对象的某个位置。它不在您粘贴的代码中,所以我猜它在超类上。

也许您可以简单地发送创建scapy.layers.dhcp.BOOTP实例所需的所有参数而不是实例以避免问题?

答案 1 :(得分:0)

另一个可能有助于诊断这类问题的方法是使用pickle模块而不是cPickle(必须由queues.py隐式使用)

我遇到了类似的情况,得到了一条完全没有用的信息,

Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

我徘徊在调试器中,发现对象被腌制,并尝试将其传递给

pickle.dump(myobj,open('outfile','w'),-1)  

并得到了更多帮助:

PicklingError: Can't pickle <function findAllRefs at 0x105809f50>: 
   it's not found as buildsys.repoclient.findAllRefs

更直接地指出了有问题的代码。

答案 2 :(得分:0)

我使用的解决方案是对数据包进行str,然后将其放入队列......