Question

我在我的一个程序中使用并行Python。当我通过CLI运行Python程序时它运行得很好。但是当我通过我的调试器运行它会抛出下一个错误

  File "D:/Unief/Thesis/deepmedic-master\deepmedic\trainValidateTestVisualiseParallel.py", line 1063, in do_training
    job_server = pp.Server(ncpus=1, ppservers=ppservers) # Creates jobserver with automatically detected number of workers
  File "D:\Anaconda2\lib\site-packages\pp-1.6.4-py2.7.egg\pp.py", line 339, in __init__
    self.set_ncpus(ncpus)
  File "D:\Anaconda2\lib\site-packages\pp-1.6.4-py2.7.egg\pp.py", line 503, in set_ncpus
    range(ncpus - len(self.__workers))])
  File "D:\Anaconda2\lib\site-packages\pp-1.6.4-py2.7.egg\pp.py", line 138, in __init__
    self.start()
  File "D:\Anaconda2\lib\site-packages\pp-1.6.4-py2.7.egg\pp.py", line 149, in start
    self.pid = int(self.t.receive())
  File "D:\Anaconda2\lib\site-packages\pp-1.6.4-py2.7.egg\pptransport.py", line 140, in receive
    raise RuntimeError("Communication pipe read error")
RuntimeError: Communication pipe read error

通过调试器运行下一个pp示例时会发生同样的错误：

import math, sys, time
import pp
import os

def isprime(n):
    """Returns True if n is prime and False otherwise"""
    if not isinstance(n, int):
        raise TypeError("argument passed to is_prime is not of 'int' type")
    if n < 2:
        return False
    if n == 2:
        return True
    max = int(math.ceil(math.sqrt(n)))
    i = 2
    while i <= max:
        if n % i == 0:
            return False
        i += 1
    return True

def sum_primes(n):
    """Calculates sum of all primes below given integer n"""
    return sum([x for x in xrange(2,n) if isprime(x)])

print """Usage: python sum_primes.py [ncpus]
    [ncpus] - the number of workers to run in parallel,
    if omitted it will be set to the number of processors in the system
"""

print sys.argv
print sys.stdin
print sys.stdout
print sys.stderr

print 'read method'

#print 'str:', sys.stdin.read()
#print 'str:', sys.stdout.read()


print 'pptest print done'

# tuple of all parallel python servers to connect with
ppservers = ()
#ppservers = ("10.0.0.1",)

if len(sys.argv) > 1:
    ncpus = int(sys.argv[1])
    # Creates jobserver with ncpus workers
    job_server = pp.Server(ncpus, ppservers=ppservers)
else:
    # Creates jobserver with automatically detected number of workers
    job_server = pp.Server(ppservers=ppservers)

print "Starting pp with", job_server.get_ncpus(), "workers"



# Submit a job of calulating sum_primes(100) for execution.
# sum_primes - the function
# (100,) - tuple with arguments for sum_primes
# (isprime,) - tuple with functions on which function sum_primes depends
# ("math",) - tuple with module names which must be imported before sum_primes execution
# Execution starts as soon as one of the workers will become available
job1 = job_server.submit(sum_primes, (100,), (isprime,), ("math",))

# Retrieves the result calculated by job1
# The value of job1() is the same as sum_primes(100)
# If the job has not been finished yet, execution will wait here until result is available
result = job1()

print "Sum of primes below 100 is", result

start_time = time.time()

# The following submits 8 jobs and then retrieves the results
inputs = (100000, 100100, 100200, 100300, 100400, 100500, 100600, 100700)
jobs = [(input, job_server.submit(sum_primes,(input,), (isprime,), ("math",))) for input in inputs]
for input, job in jobs:
    print "Sum of primes below", input, "is", job()

print "Time elapsed: ", time.time() - start_time, "s"
job_server.print_stats()


# Parallel Python Software: http://www.parallelpython.com

似乎与缺少CLI有关。

有没有人有解决方案？

Answer 1

真正的问题是parallel-python确实存在：

class PipeTransport(Transport):
    def __init__(self, r, w):
        if isinstance(r, types.FileType) and isinstance(w, types.FileType):
            self.r = r
            self.w = w
        else:
            raise TypeError("Both arguments of PipeTransport constructor " \
                    "must be file objects")

和sys.stdin由调试器进行了猴子修补。

本地修补程序正在将并行Python更改为使用sys.__stdin__（由于这些检查，它也已经使用sys.__stdout__，所以，我想这很自然）。

即：将代码更改为： self.t = pptransport.CPipeTransport(sys.__stdin__, sys.__stdout__)-代替当前的self.t = pptransport.CPipeTransport(sys.stdin, sys.__stdout__)。

在ppworker.py（在python3或python2文件夹中，具体取决于您使用的python版本）。

有关调试器方面的报告，另请参见：https://github.com/microsoft/debugpy/issues/296。

Answer 2

我遇到了同样的问题，但无法解决根本原因。作为一种解决方法，在调试时，我在'import pp'之后添加以下代码。

class FakeServer(object):
    def __init__(self,ncpus,ppservers):
        pass
     def submit(self,fn,params,localfunctions=None, externalmodules=None):
         result = apply(fn,params)
         return lambda: result
 pp.Server = FakeServer

现在您的代码是单线程的，可以轻松调试，而无需更改pp的使用。

parallel python：通信管道读取错误

2 个答案: