Question

我想加载一个pickle，它在Python 2.7中工作，但在Python 3.6中没有加载时我收到内存错误。我尝试将重新格式化的pickle改为python 3格式，然后尝试加载它，但问题仍然存在，

使用Windows 10笔记本电脑，8GB内存，pickle文件的大小为1.1GB。泡菜有14804726行和10列。

有关如何进一步解决此问题的任何线索？

Traceback (most recent call last):

  File "<ipython-input-3-3fea1d423973>", line 1, in <module>
    runfile('D:/PROJECTS/FR24/Scripts/pickletest.py', wdir='D:/PROJECTS/FR24/Scripts')

  File "c:\python36\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "c:\python36\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "D:/PROJECTS/FR24/Scripts/pickletest.py", line 11, in <module>
    readpickle(picklefile)

  File "D:/PROJECTS/FR24/Scripts/pickletest.py", line 7, in readpickle
    rawdata = pd.read_pickle(picklefile) #to load pickle

  File "c:\python36\lib\site-packages\pandas\io\pickle.py", line 180, in read_pickle
    return try_read(path, encoding='latin1')

  File "c:\python36\lib\site-packages\pandas\io\pickle.py", line 175, in try_read
    lambda f: pc.load(f, encoding=encoding, compat=True))

  File "c:\python36\lib\site-packages\pandas\io\pickle.py", line 149, in read_wrapper
    return func(f)

  File "c:\python36\lib\site-packages\pandas\io\pickle.py", line 175, in <lambda>
    lambda f: pc.load(f, encoding=encoding, compat=True))

  File "c:\python36\lib\site-packages\pandas\compat\pickle_compat.py", line 212, in load
    return up.load()

  File "c:\python36\lib\pickle.py", line 1050, in load
    dispatch[key[0]](self)

  File "c:\python36\lib\pickle.py", line 1077, in load_frame
    self._unframer.load_frame(frame_size)

  File "c:\python36\lib\pickle.py", line 257, in load_frame
    self.current_frame = io.BytesIO(self.file_read(frame_size))

MemoryError

Traceback (most recent call last):

  File "<ipython-input-3-3fea1d423973>", line 1, in <module>
    runfile('D:/PROJECTS/FR24/Scripts/pickletest.py', wdir='D:/PROJECTS/FR24/Scripts')

  File "c:\python36\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "c:\python36\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "D:/PROJECTS/FR24/Scripts/pickletest.py", line 11, in <module>
    readpickle(picklefile)

  File "D:/PROJECTS/FR24/Scripts/pickletest.py", line 7, in readpickle
    rawdata = pd.read_pickle(picklefile) #to load pickle

  File "c:\python36\lib\site-packages\pandas\io\pickle.py", line 180, in read_pickle
    return try_read(path, encoding='latin1')

  File "c:\python36\lib\site-packages\pandas\io\pickle.py", line 175, in try_read
    lambda f: pc.load(f, encoding=encoding, compat=True))

  File "c:\python36\lib\site-packages\pandas\io\pickle.py", line 149, in read_wrapper
    return func(f)

  File "c:\python36\lib\site-packages\pandas\io\pickle.py", line 175, in <lambda>
    lambda f: pc.load(f, encoding=encoding, compat=True))

  File "c:\python36\lib\site-packages\pandas\compat\pickle_compat.py", line 212, in load
    return up.load()

  File "c:\python36\lib\pickle.py", line 1050, in load
    dispatch[key[0]](self)

  File "c:\python36\lib\pickle.py", line 1077, in load_frame
    self._unframer.load_frame(frame_size)

  File "c:\python36\lib\pickle.py", line 257, in load_frame
    self.current_frame = io.BytesIO(self.file_read(frame_size))

MemoryError

错误：

NetUserChangePassword

Answer 1

所以最后我弄清楚问题是什么，它适用于python 2.7而不是python 3.6，当比较这些时我注意到python 3.6是32位而python 2.7版本是64位。所以在卸载并重新安装64位python 3.6版本后，pickle加载就好了。

无法加载Pickle文件，内存错误

1 个答案: