Question

在尝试使用Scipy的优化算法来最小化在子过程中计算其值的函数时，我发现基于梯度的算法（到目前为止的流域购物和L-BFGS-B）遇到以下错误：优化线562 of optimize.py：

grad[k] = (f(*((xk + d,) + args)) - f0) / d[k]
TypeError：不支持的操作数类型 - ：＆＃39; NoneType＆＃39;和＆＃39; NoneType＆＃39;

以下是生成此错误的代码的简单示例：

import multiprocessing as mp
from scipy.optimize import basinhopping

def runEnvironment(x):
    return x**2

def func(x):
    if __name__ == '__main__':
        print "x:",x
        pool = mp.Pool(processes=1)

        results=pool.apply(runEnvironment,(x,))
        pool.close()
        return results

x0=5    
ret=basinhopping(func, x0, niter=100, T=1.0, stepsize=0.1, minimizer_kwargs=None, take_step=None, accept_test=None, callback=None, interval=50, disp=False, niter_success=None)

请注意，如果删除了多处理组件，或者使用了基于非梯度的算法（如COBYLA），则此代码可以正常运行。任何人都可以想到这种情况发生的原因吗？

Answer 1

你的if __name__ == '__main__':成语位置不正确 - 以这种方式重新排列：

import multiprocessing as mp
from scipy.optimize import basinhopping

def runEnvironment(x):
    return x**2

def func(x):

    print "x:",x
    pool = mp.Pool(processes=1)

    results=pool.apply(runEnvironment,(x,))
    pool.close()
    return results

if __name__ == '__main__':
    x0=5
    ret=basinhopping(func, x0, niter=100, T=1.0, stepsize=0.1, minimizer_kwargs=None, take_step=None, accept_test=None, callback=None, interval=50, disp=False, niter_success=None)

Answer 2

每个只有一个工人创建许多小mp.Pool是低效的处理。因此，每次调用func时创建一个池也是低效的多次调用func。

相反，在程序开始时创建一个 Pool，并将池传递给每次调用func：

if __name__ == '__main__':
    pool = mp.Pool()
    x0=5    
    ret = optimize.basinhopping(
        func, x0, niter=100, T=1.0, stepsize=0.1,
        minimizer_kwargs=dict(args=pool), 
        take_step=None, accept_test=None, callback=None,
        interval=50, disp=False, niter_success=None)
    pool.close()
    print(ret)

minimizer_kwargs=dict(args=pool)告诉optimize.basinhopping将pool作为func的附加参数传递。

您也可以使用

logger = mp.log_to_stderr(logging.INFO)

获取日志语句，显示调用函数的过程。例如，

import multiprocessing as mp
from scipy import optimize
import logging
logger = mp.log_to_stderr(logging.INFO)

def runEnvironment(x):
    logger.info('runEnvironment({}) called'.format(x))
    return x**2

def func(x, pool):
    logger.info('func({}) called'.format(x))
    results = pool.apply(runEnvironment,(x,))
    return results

if __name__ == '__main__':
    pool = mp.Pool()
    x0=5    
    ret = optimize.basinhopping(
        func, x0, niter=100, T=1.0, stepsize=0.1,
        minimizer_kwargs=dict(args=pool), 
        take_step=None, accept_test=None, callback=None,
        interval=50, disp=False, niter_success=None)
    pool.close()
    print(ret)

打印

[INFO/PoolWorker-1] child process calling self.run()
[INFO/PoolWorker-2] child process calling self.run()
[INFO/PoolWorker-3] child process calling self.run()
[INFO/PoolWorker-4] child process calling self.run()
[INFO/MainProcess] func([ 5.]) called
[INFO/PoolWorker-1] runEnvironment([ 5.]) called
[INFO/MainProcess] func([ 5.00000001]) called
[INFO/PoolWorker-2] runEnvironment([ 5.00000001]) called
[INFO/MainProcess] func([ 5.]) called
[INFO/PoolWorker-3] runEnvironment([ 5.]) called
[INFO/MainProcess] func([-5.]) called
[INFO/PoolWorker-4] runEnvironment([-5.]) called

这表明func始终由主进程调用，而 runEnvironment由工作进程运行。

请注意，对func的调用会按顺序发生。从中获取任何好处 pool，每次调用func时，您都必须使用更多处理器。

Scipy的优化与多处理不兼容？

2 个答案: