使用带有类的futures.concurrent

时间:2016-01-05 03:18:37

标签: python concurrent.futures

我有一个如下所示的主要内容:

import gather_filings_per_filer
import os
from concurrent import futures

def put_last_i(arg):
    with open(os.path.dirname(os.path.abspath(__file__)) + '/last_i.txt','w') as f:
            f.write(arg)

def get_last_i():
    with open(os.path.dirname(os.path.abspath(__file__)) + '/last_i.txt','r') as f:
            data = f.readlines()
        return data[0]

if __name__=='__main__':
    i = int(get_last_i())
    g_f_p_f = gather_filings_per_filer.gather_filings_per()
    jobs=[]


    with futures.ThreadPoolExecutor(max_workers = 3) as executor:
        my_d = dict((executor.submit(g_f_p_f.mt_main),j) for j in range(i,297085))

        for future in futures.as_completed(my_d):
            print future
            print future.exception()

g_f_p_f.mt_main如下所示:

class gather_filings_per:
       def mt_main(self, i):
            print "Started:",i
            self.w_m = wrap_mysql.wrap_mysql()
            flag = True
            #do stuff...

给我以下结果:

<Future at 0x5dc2710 state=finished raised TypeError>
mt_main() takes exactly 2 arguments (1 given)
<Future at 0x5dc25b0 state=finished raised TypeError>
mt_main() takes exactly 2 arguments (1 given)

从我的角度来看mt_main只需要1个参数(鉴于典型的self行为,不应该要求自己。)

我错过了什么似乎出错?

1 个答案:

答案 0 :(得分:3)

你是正确的,除了隐含的self之外,你只需要提供一个额外的参数。但你没有给予任何帮助。所以,你是一个简短的。拆分提交以使其在视觉上清晰:

my_d = dict(  # Making a dict
            # With key as result object from invoking `mt_main` with no additional arguments
            (executor.submit(g_f_p_f.mt_main),
            # And value j
             j)
            for j in range(i,297085))

也许您打算将j作为参数传递?假设它也应该是值,那就是:

# Note second argument to submit, which becomes only non-self argument to mt_main
my_d = dict((executor.submit(g_f_p_f.mt_main, j),j) for j in range(i,297085))

或者为了简化时态,因为concurrent.futures应该意味着您可以使用dict comprehensionssubmit也会将:来电与my_d = {executor.submit(g_f_p_f.mt_main, j): j for j in range(i, 297085)} 的值分开为了更好的可视化解析):

{{1}}