Python:在torch.utils.data.DataLoader对象上调用iter时出现BrokenPipeError

时间:2018-06-13 20:20:29

标签: python python-3.x pytorch broken-pipe

我目前正在关注PyTorch教程,我正试图让this example工作。然而,在运行第dataiter = iter(trainloader)行时,我的程序首先做了一些事我不明白它是怎么可能的 - 它似乎在程序中向后跳,并重新运行它已经运行的代码行,即使没有环。然后它会遇到BrokenPipeError并崩溃。

以下是触发问题的最小工作示例:

import torch
import torchvision
import torchvision.transforms as transforms

transform = transforms.Compose([transforms.ToTensor(),
                                transforms.Normalize((0.5, 0.5, 0.5),
                                                     (0.5, 0.5, 0.5))])
trainset = torchvision.datasets.CIFAR10(root='./data',
                                        train=True,
                                        download=True,
                                        transform=transform)
trainloader = torch.utils.data.DataLoader(trainset,
                                          batch_size=4,
                                          shuffle=True,
                                          num_workers=2)
dataiter = iter(trainloader)

这是我得到的输出:

"D:\Program Files\Python35\python.exe" D:/work/programs/pytorch/60_minute_blitz_tutorial/3_training_a_classifier/MWE.py
Files already downloaded and verified
Files already downloaded and verified
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\Program Files\Python35\lib\multiprocessing\spawn.py", line 106, in spawn_main
    exitcode = _main(fd)
  File "D:\Program Files\Python35\lib\multiprocessing\spawn.py", line 115, in _main
    prepare(preparation_data)
  File "D:\Program Files\Python35\lib\multiprocessing\spawn.py", line 226, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "D:\Program Files\Python35\lib\multiprocessing\spawn.py", line 278, in _fixup_main_from_path
    run_name="__mp_main__")
  File "D:\Program Files\Python35\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "D:\Program Files\Python35\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "D:\Program Files\Python35\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\work\programs\pytorch\60_minute_blitz_tutorial\3_training_a_classifier\MWE.py", line 16, in <module>
    dataiter = iter(trainloader)
  File "D:\Program Files\Python35\lib\site-packages\torch\utils\data\dataloader.py", line 451, in __iter__
    return _DataLoaderIter(self)
  File "D:\Program Files\Python35\lib\site-packages\torch\utils\data\dataloader.py", line 239, in __init__
    w.start()
  File "D:\Program Files\Python35\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "D:\Program Files\Python35\lib\multiprocessing\context.py", line 212, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "D:\Program Files\Python35\lib\multiprocessing\context.py", line 313, in _Popen
    return Popen(process_obj)
  File "D:\Program Files\Python35\lib\multiprocessing\popen_spawn_win32.py", line 34, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "D:\Program Files\Python35\lib\multiprocessing\spawn.py", line 144, in get_preparation_data
    _check_not_importing_main()
  File "D:\Program Files\Python35\lib\multiprocessing\spawn.py", line 137, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
Traceback (most recent call last):
  File "D:/work/programs/pytorch/60_minute_blitz_tutorial/3_training_a_classifier/MWE.py", line 16, in <module>
    dataiter = iter(trainloader)
  File "D:\Program Files\Python35\lib\site-packages\torch\utils\data\dataloader.py", line 451, in __iter__
    return _DataLoaderIter(self)
  File "D:\Program Files\Python35\lib\site-packages\torch\utils\data\dataloader.py", line 239, in __init__
    w.start()
  File "D:\Program Files\Python35\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "D:\Program Files\Python35\lib\multiprocessing\context.py", line 212, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "D:\Program Files\Python35\lib\multiprocessing\context.py", line 313, in _Popen
    return Popen(process_obj)
  File "D:\Program Files\Python35\lib\multiprocessing\popen_spawn_win32.py", line 66, in __init__
    reduction.dump(process_obj, to_child)
  File "D:\Program Files\Python35\lib\multiprocessing\reduction.py", line 59, in dump
    ForkingPickler(file, protocol).dump(obj)
BrokenPipeError: [Errno 32] Broken pipe

Process finished with exit code 1

输出“Files already downloaded and verified”只应打印一次 - 创建trainset时 - 但打印两次。

这是为什么?为什么我得到BrokenPipeError?最重要的是,我该如何解决这个问题?

我正在运行Windows 10,Python 3.5.3,CUDA 8.0.60以及运行时安装的PyTorch 0.4.0

pip install http://download.pytorch.org/whl/cu80/torch-0.4.0-cp35-cp35m-win_amd64.whl
pip install torchvision

按照PyTorch webpage的指示(好吧,他们使用pip3代替pip,但这不应该有任何区别,因为Python 3.5是我安装的唯一Python版本)

0 个答案:

没有答案