Question

我正在Bash工作。我有一系列嵌套的for循环，用于迭代查找是否存在96个条形码序列的三个列表。我的目标是找到条形码的每个唯一组合，其中有96x96x96（884,736）个可能的组合。

string regular = "r";
string premium = "p";
Console.WriteLine("Please enter your service (r or p:");
var input = Console.ReadLine();
if (input == regular )
{
    //your reg logic here
}
else if (input == premium)
{
   // your premium logic here
}
else
{
   //handle case that none of above entered
}

此代码有效，并且我能够成功提取每个条形码组合的序列。但是，我认为可以通过并行化此循环结构来提高处理大型文件的速度。我知道我可以使用GNU并行执行此操作，但是我正在努力嵌套并行化。

for barcode1 in "${ROUND1_BARCODES[@]}";
do
grep -B 1 -A 2 "$barcode1" $FASTQ_R > ROUND1_MATCH.fastq
echo barcode1.is.$barcode1 >> outputLOG

    if [ -s ROUND1_MATCH.fastq ]
    then

        # Now we will look for the presence of ROUND2 barcodes in our reads containing barcodes from the previous step
        for barcode2 in "${ROUND2_BARCODES[@]}";
        do
        grep -B 1 -A 2 "$barcode2" ROUND1_MATCH.fastq > ROUND2_MATCH.fastq

            if [ -s ROUND2_MATCH.fastq ]
            then

                # Now we will look for the presence of ROUND3 barcodes in our reads containing barcodes from the previous step 
                for barcode3 in "${ROUND3_BARCODES[@]}";
                do
                grep -B 1 -A 2 "$barcode3" ./ROUND2_MATCH.fastq | sed '/^--/d' > ROUND3_MATCH.fastq

                # If matches are found we will write them to an output .fastq file itteratively labelled with an ID number
                if [ -s ROUND3_MATCH.fastq ]
                then
                mv ROUND3_MATCH.fastq results/result.$count.2.fastq
                fi

                count=`expr $count + 1` 
                done
            fi
        done
    fi
done

如何使用并行成功地重新创建for循环的嵌套结构？

Answer 1

仅并行化内部循环：

for barcode1 in "${ROUND1_BARCODES[@]}";
do
grep -B 1 -A 2 "$barcode1" $FASTQ_R > ROUND1_MATCH.fastq
echo barcode1.is.$barcode1 >> outputLOG

    if [ -s ROUND1_MATCH.fastq ]
    then

        # Now we will look for the presence of ROUND2 barcodes in our reads containing barcodes from the previous step
        for barcode2 in "${ROUND2_BARCODES[@]}";
        do
        grep -B 1 -A 2 "$barcode2" ROUND1_MATCH.fastq > ROUND2_MATCH.fastq
            if [ -s ROUND2_MATCH.fastq ]
            then
                # Now we will look for the presence of ROUND3 barcodes in our reads containing barcodes from the previous step 
                doit() {
                    grep -B 1 -A 2 "$1" ./ROUND2_MATCH.fastq | sed '/^--/d'
                }
                export -f doit
                parallel -j0 doit {} '>' results/$barcode1-$barcode2-{} ::: "${ROUND3_BARCODES[@]}"
                # TODO remove files with 0 length
            fi
        done
    fi
done

使用GNU Parallel并行化嵌套循环

1 个答案: