适用于Python的SGE数组作业从循环获取输入

时间:2018-07-12 17:59:40

标签: python bash shell cluster-computing

我正在尝试提交应该获得不同输入的Python脚本的数组作业。现在,我只是像这样分别提交每个工作:

#!/bin/bash

P_vec=($(seq 0.92 0.01 1.15))
length_P=${#P_vec[@]}

for (( i=0; i<${length_P}; i++ ));
do
    qsub -cwd python saver.py -s ${P_vec[i]}
done

但是,我想像在此tutorial中找到的那样,将其作为数组作业提交,其中的基本示例是:

#!/bin/sh
# Tell the SGE that this is an array job, with "tasks" to be numbered 1 to 10000
#$ -t 1-10000
# When a single command in the array job is sent to a compute node,
# its task number is stored in the variable SGE_TASK_ID,
# so we can use the value of that variable to get the results we want:
~/programs/program -i ~/data/input.$SGE_TASK_ID -o ~/results/output.$SGE_TASK_ID

如何在上一个脚本的循环内合并-t标志?

尝试以下方法:

#!/bin/bash


P_vec=($(seq 0.92 0.01 1.15))
length_P=${#P_vec[@]}
#$ -t 1-$length_P

python saver.py -s ${P_vec[$SGE_TASK_ID]}

给出错误消息:

Unable to read script file because of error: Numerical value invalid!
The initial portion of string "$length_P" contains no decimal number

虽然没有给出任何错误消息,但没有提供所需的输出:

#!/bin/bash
#$ -t 1-3

python saver.py -s $SGE_TASK_ID

1 个答案:

答案 0 :(得分:1)

无法测试,但是我的常识说:

#!/bin/bash
# Tell the SGE that this is an array job, with "tasks" to be numbered 0 to length_P-1
#$ -t 0-???????

P_vec=($(seq 0.92 0.01 1.15))

python saver.py -s ${P_vec[$SGE_TASK_ID]}