Question

我目前正在尝试使用Python多处理程序包来使CPU绑定进程更快地运行。我有一个非常大的numpy矩阵，并希望使用Pool和apply_async分割工作来计算矩阵中的值。但是，当我在函数上运行单元测试以测试它是否有效时，我得到错误＆＃34; NameError：全局名称＆＃39; self＆＃39;未定义＆＃34;。我无法在Google或StackOverflow上找到任何有用的内容。知道为什么会这样吗？

Pytest输出：

_____________________ TestBuildEMMatrix.test_build_em_matrix_simple _____________________

self = <mixemt_master.mixemt2.preprocess_test.TestBuildEMMatrix testMethod=test_build_em_matrix_simple>

    def test_build_em_matrix_simple(self):
            reads = ["1:A,2:C", "1:T,2:C", "3:T,4:T", "2:A,4:T"]
            in_mat = preprocess.build_em_matrix(self.ref, self.phy,
>                                                                                   reads, self.haps, self.args)

preprocess_test.py:272:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
preprocess.py:239: in build_em_matrix
    results[i] = results[i].get()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <multiprocessing.pool.ApplyResult object at 0x7f4218ea07d0>, timeout = None

    def get(self, timeout=None):
        self.wait(timeout)
        if not self._ready:
            raise TimeoutError
        if self._success:
            return self._value
        else:
>           raise self._value
E           NameError: global name 'self' is not defined

/vol/hpc/apps/python-anaconda2-4.3.1-abat/install/lib/python2.7/multiprocessing/pool.py:567: NameError
--------------------------------- Captured stdout call ----------------------------------
False

相关的Python函数：

def build_em_matrix_process(markers, haplogroups, pos_obs, mut_prob, column_length, start_index, end_index):

    columns = [[prob_for_vars(markers, haplogroups[j], pos_obs, mut_prob) for j in xrange(column_length)]
        for i in xrange(start_index, end_index)]

    return columns

def build_em_matrix(refseq, phylo, reads, haplogroups, args):   
    """
    Returns the matrix that describes the probabiliy of each read
    originating in each haplotype.
    """
    hvb_mat = HapVarBaseMatrix(refseq, phylo)
    read_hap_mat = numpy.empty((len(reads), len(haplogroups)))

    if args.verbose:
        sys.stderr.write('Building EM input matrix...\n')

    num_processors = args.p

    pool = Pool(processes = num_processors);
    results = []
    partition_size = int(math.ceil(len(reads) / float(num_processors)))

    for i in xrange(num_processors):
        start_index = i * partition_size
        end_index = (i + 1) * partition_size
        pos_obs = pos_obs_from_sig(reads[i])

        results.append(pool.apply_async(build_em_matrix_process, (hvb_mat.markers, haplogroups, pos_obs, hvb_mat.mut_prob, len(haplogroups), start_index, end_index)))

    column = 0
    for i in xrange(num_processors):
        results[i].wait()
        print results[i].successful()
        results[i] = results[i].get()
        for j in xrange[len(results)]:
            read_hap_mat[column] = results[i][j]
            column += 1

    if args.verbose:
        sys.stderr.write('Done.\n\n')

    return read_hap_mat

在调用＆＃39; results [i] .wait（）]之后添加了一个语句＆quot;打印结果[I] .successful（）＆＃39;，它将False打印到stdout。我不确定为什么没有返回true，因为我无法在build_em_matrix_process中找到任何错误。

Answer 1

我更多地探讨了代码，并找到了答案！

我重构了一个类的实例方法，它被build_em_matrix_process调用，是一个顶级方法，以实现这一目标。事实证明，我不小心在方法的主体中留下了对自我的引用。当我运行测试时，错误似乎来自ApplyResult.get（）本身的代码，而不是被调用的顶级方法中的代码。

Python多处理 - ApplyResult.get（）NameError：全局名称＆＃39; self＆＃39;没有定义

1 个答案: