Question

我正在尝试建立一条蛇形管道，并且它抱怨缺少MissingOutputException：

Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cluster nodes: 8
Job counts:
        count   jobs
        1       add_replace_readgroups
        1       all
        1       annotate_nirvana
        1       apply_bqsr
        1       base_recalibrator
        1       bwa_aln1
        1       bwa_aln2
        1       bwa_sampe
        1       flagstats
        1       haplotypecaller
        1       intervals
        1       isize
        1       json2vcf
        1       mark_dups
        1       mod_sam
        1       region_coverage
        1       sam2bam
        17

[Wed Jul 17 09:03:04 2019]
rule bwa_aln2:
    input: C052007W-B.2.fq.gz, /mnt/storage/projects/hiv_data/refs/PKD_GANAB/PKD.fasta
    output: tmp/C052007W-B.2.sai
    log: logs/bwa_aln2/C052007W-B.log
    jobid: 8
    wildcards: sample=C052007W-B
    threads: 8

bwa aln -t 8 -q 15 -e 50 -f tmp/C052007W-B.2.sai /mnt/storage/projects/hiv_data/refs/PKD_GANAB/PKD.fasta C052007W-B.2.fq.gz 2> logs/bwa_aln2/C052007W-B.log
Submitted job 8 with external jobid 'Submitted batch job 283790'.

[Wed Jul 17 09:03:04 2019]
rule bwa_aln1:
    input: C052007W-B.1.fq.gz, /mnt/storage/projects/hiv_data/refs/PKD_GANAB/PKD.fasta
    output: tmp/C052007W-B.1.sai
    log: logs/bwa_aln1/C052007W-B.log
    jobid: 7
    wildcards: sample=C052007W-B
    threads: 8

bwa aln -t 8 -q 15 -e 50 -f tmp/C052007W-B.1.sai /mnt/storage/projects/hiv_data/refs/PKD_GANAB/PKD.fasta C052007W-B.1.fq.gz 2> logs/bwa_aln1/C052007W-B.log
Submitted job 7 with external jobid 'Submitted batch job 283791'.
Waiting at most 30 seconds for missing files.
MissingOutputException in line 63 of /mnt/storage/home/kimy/projects/automate_CP/scripts/Snakefile:
Missing files after 30 seconds:
tmp/C052007W-B.2.sai
This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait.
Waiting at most 30 seconds for missing files.
[Wed Jul 17 09:04:04 2019]
Finished job 7.
1 of 17 steps (6%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /mnt/storage/home/kimy/projects/automate_CP/CP0340/.snakemake/log/2019-07-17T090304.049684.snakemake.log

我正在使用以下命令行运行它：

/mnt/storage/home/kimy/software/miniconda3/bin/snakemake \
    -s /mnt/storage/home/kimy/projects/automate_CP/scripts/Snakefile \
    -j 8 \
    -p \
    --latency-wait 30 \
    --configfile ./${sample}_config.yaml \
    --cluster "sbatch --job-name=NILES --ntasks=1 --partition=long --time=12:00:00 --mem=25000 --reservation=compute006"

但是，Snakemake抱怨的文件确实存在，如果我重新运行相同的snakefile，则会运行下一条规则，但会抱怨该下一条规则的输出。另一个细节：我为整个文件夹（当然，我有一堆fastqs并使用另一个配置文件）运行了管道，并且除其中两个样本外，每个样本都工作正常，促使我研究样本。

您知道我为什么会收到此错误吗？

Answer 1

是的@Colin，您是对的。 2个样本始终失败，并且增加等待时间参数解决了该问题。我以前将其增加到30秒，但似乎还不够。 60秒解决了这个问题。

Snakemake：输出确实存在时MissingOutputException

1 个答案: