使用python子进程启动LSF作业并等待它们完成

Question

很抱歉问这个长问题，

目标“”使用python子过程模块启动bsub LSF作业。用例是前几个LSF作业必须并行进行，而最后一个LSF作业则等待所有作业完成。

我尝试了以下两种方法：

方法1

Test.py

import argparse as argObj
import time 
if __name__ == "__main__":
    """
    Main function to start the script execution
    """
    argList = argObj.ArgumentParser()
    argList.add_argument("-instance_id",                             
                           required=True,
                      type =int, 
                      default=0

                    )

args = argList.parse_args()
data = ""
if args.instance_id <  1000000000 : 
    file_name =("temp/thread_%s"%(args.instance_id)) 
    data ="Running thread %s \n" % args.instance_id
else:
    file_name =("temp/thread_final") 
    data= "Running thread %s \n" % "final"
file = open (file_name,"w") 

file.write(data)
data= "Started the wait %s \n" % time.asctime( time.localtime(time.time()) )
file.write(data)
time.sleep(30)
data= "Finished the wait %s \n" % time.asctime( time.localtime(time.time()) )
file.write(data)
file.close() 

上面的脚本是从下面的函数调用的：

import subprocess 
def test(): 
    l=[]
    for i in range (0,2) : 
        command  = ('bsub -J"parent%s" -o /dev/null -q short python test.py -instance_id %s' % (str(i) , str(i)))

        process = subprocess.Popen(command,shell=True) 
        l.append(process)
    for i in l : 
        i.wait() 

    command  = ('bsub -K -J"Final" -o /dev/null -q short  python test.py -instance_id 10000001000; wait')
    subprocess.call(command,shell=True)

test()

在这种情况下，在三个输出文件中输出的输出是：（lsf作业thread_final应该在线程0和线程1完成之后启动）

Running thread 0 
Started the wait Tue Aug  7 22:41:50 2018 
Finished the wait Tue Aug  7 22:42:20 2018 

Running thread 1 
Started the wait Tue Aug  7 22:41:49 2018 
Finished the wait Tue Aug  7 22:42:19 2018 

Running thread final 
Started the wait Tue Aug  7 22:41:49 2018 
Finished the wait Tue Aug  7 22:42:19 2018 

方法2

import subprocess 
    def test(): 
        l=[]
        for i in range (0,2) : 
            command  = ('bsub -J"parent%s" -o /dev/null -q short python test.py -instance_id %s' % (str(i) , str(i)))

        process = subprocess.Popen(command,shell=True) 
        l.append(process)


    command  = ('bsub -K -J"Final" -o /dev/null -w "ended(parent*)"-q short  python test.py -instance_id 10000001000; wait')
    subprocess.call(command,shell=True)
 for i in l : 
        i.wait() 
test()

在这种情况下，我看到有时Thread final会按预期执行，并且大多数情况下它会在任何父线程启动之前执行。 我可以得到一些有关如何实现这一目标的指导吗？

谢谢！