python冻结中的并行处理

时间:2019-08-30 17:50:23

标签: python-3.x python-multiprocessing

我对Python相当陌生。我已经看过其他文章,但是由于我正在使用带有多个参数的函数,因此无法成功将其应用于我的代码。当我运行它时,它将运行几秒钟,然后停止。最终它将结束,但是Spyder将不会再运行其他任何东西,而必须重新启动。我看到过类似的帖子,发现队列已满,但是我无法成功地将这些解决方案应用于此。抱歉,我现在想不出一种方法来使其更具可重复性。只是想知道是否有明显的问题。预先感谢您的帮助。

indices = random.sample(range(X_test.shape[0]), 50)
# tried this didnt work -> manager=mp.Manager()
# output=manager.Queue()
output=Queue()

def mpgs(observation, X, y, treat, cat, propensityIndex, outcomeIndex, outcomeTrt, outcomeCtrl, idx, score_function):
        ites = gsf.gs(observation, X, y, treat, cat, propensityIndex, outcomeIndex, outcomeTrt, outcomeCtrl, idx, score_function)
        output.put(ites)

# Set up a list of processes that we want to run
processes = [mp.Process(target=mpgs, args=(obs_to_interprete, X_test, y_test, trt_test, ['x1', 'x6', 'x9'], np.r_[0:10], np.r_[0:10], rf_t, rf_c, idx, pred)) for idx in indices]

# Run processes
for p in processes:
    p.start()

# Exit the completed processes
for p in processes:
    p.join()

# Get process results from the output queue
results = [output.get() for p in processes]

主要的gs()函数(如果相关):

def gs(observation, X, y, treat, cat, propensityIndex, outcomeIndex, outcomeTrt, outcomeCtrl, idx, score_function):
    scores_c, nearest_ennemy1_c, uni_layer_c = interpretability_metrics(X=X, treat=treat, score_function=score_function, obs_to_interprete=observation, propensityIndex = propensityIndex, outcomeIndex=outcomeIndex, cat=cat, idx=idx)
    key_min = min(scores_c.keys(), key=(lambda k: scores_c[k]['distance']))
    print('Minimum distance:', scores_c[key_min]['distance'])
    adversary = nearest_ennemy1_c[key_min]
    print('Nearest adversary:', adversary)
    print('Nearest adversary propensity score:', score_function(adversary[propensityIndex]))
    print('Original observation propensity score:',      score_function(pd.DataFrame(observation).iloc[propensityIndex].values))
    # Individual treatment effect  
    treated = treat[idx]
    def ite(original, counterfactual, treated, idx):
        if treated==1:
            y_trt=y[idx]
            y_c=outcomeCtrl.predict(counterfactual[outcomeIndex].reshape(1, -1))[0]
        elif treated==0:
            y_c=y[idx]
            y_trt=outcomeTrt.predict(counterfactual[outcomeIndex].reshape(1, -1))[0]
        else: 
            print("Treated variable doesn't take on 0 or 1.")
        return y_trt-y_c
    ITE = ite(observation, adversary, treated=treated, idx=idx)
    return ITE

0 个答案:

没有答案