Question

我是Snakemake的新手，我正在尝试开发一些管道。使用通配符时，我会遇到一些问题，尝试尽可能自动执行生物信息学分析。当管道变得更加复杂时，我遇到了麻烦（如下所示）。看来Snakemake无法正确解析通配符。在Snakefile的空运行期间，通配符值在某些规则的执行中看起来正确。但是，相同的通配符会在管道的不同步骤（规则）中导致错误，我无法弄清原因。在下面，我提供了空运行的代码和输出消息。

num=["327905-LR-41624_normal","327907-LR-41624_tumor"]
num_normal=["327905-LR-41624"]
num_tumor=["327907-LR-41624"]

path="/path/to/Snakemake/"
genome="/path/to/references_genome/Mus_musculus.GRCm38.dna_rm.toplevel.fa"

rule all:
    input:  
    expand("/path/to/Snakemake/AS-{num_tum}_tumor_no_dupl_sort_RG_LB.bam",num_tum=num_tumor),
    expand("/path/to/Snakemake/AS-{num_norm}_normal_no_dupl_sort_RG_LB.bam",num_norm=num_normal)
ruleorder: samtools_sort > remove_duplicates >  samtools_index #>     add_readgroup_tumor > add_readgroup_normal

rule trim_galore:
    input:
        r1="/path/to/Snakemake/AS-{num}_R1.fastq",
        r2="/path/to/Snakemake/AS-{num}_R2.fastq"
    output:
        "/path/to/Snakemake/AS-{num }_R1_val_1.fq",
        "/path/to/Snakemake/AS-{num }_R2_val_2.fq"
    shell:
        "module load trim-galore/0.5.0 ; module load pypy/2.7-6.0.0 ; trim_galore  --output_dir /path/to/Snakemake/  --paired {input.r1} {input.r2}  "  

rule bwa_mem:
    input:
        R1="/path/to/Snakemake/AS-{num}_R1_val_1.fq",
        R2="/path/to/Snakemake/AS-{num}_R2_val_2.fq"
    output:
        "/path/to/Snakemake/AS-{num}.bam"
    shell:
        "module load samtools/default ; module load bwa/0.7.8 ; bwa mem  {genome}  {input.R1} {input.R2} | samtools view -h -b  > {output} "

rule samtools_sort:
    input:
        "/path/to/Snakemake/AS-{num}.bam"
    output:
        "/path/to/Snakemake/AS-{num}_sort.bam"
    shell:
        "module load samtools/default ; samtools sort -n  -O BAM {input} > {output} "

rule remove_duplicates:
    input:
        "/path/to/Snakemake/AS-{num}_sort.bam"
    output:
        outbam="/path/to/Snakemake/AS-{num}_no_dupl_sort.bam",
        metrics="/path/to/Snakemake/AS-{num}_dupl_metrics.txt"
    shell:
        "module load gatk/4.0.9.0 ; gatk MarkDuplicates -I {input}  -O {output.outbam} -M {output.metrics}  --REMOVE_DUPLICATES=true "

rule samtools_index:
    input:
        "/path/to/Snakemake/AS-{num}_no_dupl_sort.bam"
    output:
        "/path/to/Snakemake/AS-{num}_no_dupl_sort.bam.bai"
    shell:
        "module load samtools/default ; samtools index  {input} "

rule add_readgroup_normal:
    input:
    "/path/to/Snakemake/AS-{num_normal}_normal_no_dupl_sort.bam"
output:
    "/path/to/Snakemake/AS-{num_normal}_normal_no_dupl_sort_RG_LB.bam"
shell:
    "module load gatk/4.0.9.0 ;  gatk AddOrReplaceReadGroups   -PL Illumina -LB  { num_normal }   -PU  { num_normal }  -SM  NORMAL  -I  { input }    -O  {output} "

rule add_readgroup_tumor:
    input:
        "/path/to/Snakemake/AS-{num_tumor}_tumor_no_dupl_sort.bam"
    output:
        "/path/to/Snakemake/AS-{num_tumor}_tumor_no_dupl_sort_RG_LB.bam"
    shell:
        "module load gatk/4.0.9.0 ;  gatk AddOrReplaceReadGroups   -PL Illumina -LB  { num_tumor }   -PU  { num_tumor }  -SM  TUMOR     -I  { input }    -O  {output} "

当我使用以下命令测试Snakefile时： .local / bin / snakemake -s Snakefile_pipeline --dryrun

我得到以下信息：

**Building DAG of jobs...**


**Job counts:**
    **count jobs
    1   add_readgroup_normal
    1   add_readgroup_tumor
    1   all
    2   bwa_mem
    2   remove_duplicates
    2   samtools_sort
    2   trim_galore
    11**

**[Mon Apr  8 16:14:27 2019]
rule trim_galore:
    input: /path/to/Snakemake/AS-327907-LR-41624_tumor_R1.fastq, /path/to/Snakemake/AS-327907-LR-41624_tumor_R2.fastq
    output: /path/to/Snakemake/AS-327907-LR-41624_tumor_R1_val_1.fq, /path/to/Snakemake/AS-327907-LR-41624_tumor_R2_val_2.fq
    jobid: 9
    wildcards: num=327907-LR-41624_tumor**


**[Mon Apr  8 16:14:27 2019]
rule trim_galore:
    input: /path/to/Snakemake/AS-327905-LR-41624_normal_R1.fastq, /path/to/Snakemake/AS-327905-LR-41624_normal_R2.fastq
    output: /path/to/Snakemake/AS-327905-LR-41624_normal_R1_val_1.fq, /path/to/Snakemake/AS-327905-LR-41624_normal_R2_val_2.fq
    jobid: 10
    wildcards: num=327905-LR-41624_normal**


**[Mon Apr  8 16:14:27 2019]
rule bwa_mem:
    input: /path/to/Snakemake/AS-327905-LR-41624_normal_R1_val_1.fq, /path/to/Snakemake/AS-327905-LR-41624_normal_R2_val_2.fq
    output: /path/to/Snakemake/AS-327905-LR-41624_normal.bam
    jobid: 8
    wildcards: num=327905-LR-41624_normal**


**[Mon Apr  8 16:14:27 2019]
rule bwa_mem:
    input: /path/to/Snakemake/AS-327907-LR-41624_tumor_R1_val_1.fq, /path/to/Snakemake/AS-327907-LR-41624_tumor_R2_val_2.fq
    output: /path/to/Snakemake/AS-327907-LR-41624_tumor.bam
    jobid: 7
    wildcards: num=327907-LR-41624_tumor**


**[Mon Apr  8 16:14:27 2019]
rule samtools_sort:
    input: /path/to/Snakemake/AS-327907-LR-41624_tumor.bam
    output: /path/to/Snakemake/AS-327907-LR-41624_tumor_sort.bam
    jobid: 5
    wildcards: num=327907-LR-41624_tumor**


**[Mon Apr  8 16:14:27 2019]
rule samtools_sort:
    input: /path/to/Snakemake/AS-327905-LR-41624_normal.bam
    output: /path/to/Snakemake/AS-327905-LR-41624_normal_sort.bam
    jobid: 6
    wildcards: num=327905-LR-41624_normal**


**[Mon Apr  8 16:14:27 2019]
rule remove_duplicates:
    input: /path/to/Snakemake/AS-327907-LR-41624_tumor_sort.bam
    output: /path/to/Snakemake/AS-327907-LR-41624_tumor_no_dupl_sort.bam, /path/to/Snakemake/AS-327907-LR-41624_tumor_dupl_metrics.txt
    jobid: 3
    wildcards: num=327907-LR-41624_tumor**


**[Mon Apr  8 16:14:27 2019]
rule remove_duplicates:
    input: /path/to/Snakemake/AS-327905-LR-41624_normal_sort.bam
    output: /path/to/Snakemake/AS-327905-LR-41624_normal_no_dupl_sort.bam, /path/to/Snakemake/AS-327905-LR-41624_normal_dupl_metrics.txt
    jobid: 4
    wildcards: num=327905-LR-41624_normal**


**[Mon Apr  8 16:14:27 2019]
rule add_readgroup_normal:
    input: /path/to/Snakemake/AS-327905-LR-41624_normal_no_dupl_sort.bam
    output: /path/to/Snakemake/AS-327905-LR-41624_normal_no_dupl_sort_RG_LB.bam
    jobid: 2
    wildcards: num_normal=327905-LR-41624**

**RuleException in line 93 of /home/l136n/Snakefile_mapping_snv_call_pipeline2:
NameError: The name ' num_normal ' is unknown in this context. Please make sure that you defined that variable. Also note that braces not used for variable access have to be escaped by repeating them, i.e. {{print $1}}**

我已经搜索了该错误，但发现的帮助很小。另外，我再次检查了管道是否存在混乱。我期望的输出在规则“全部”中指示。规则“ add_readgroup_normal”和“ add_readgroup_tumor”应采用由先前步骤生成的输入文件的不同子集，这些子集将在所有文件上运行。我想知道是否由于将此分为两个子集而以某种方式出现了问题。我再说一遍，我对Snakemake还是很陌生，所以我可能在某个地方缺少一些愚蠢的东西！任何帮助将不胜感激，因为我完全陷入困境！提前非常感谢您！

num=["327905-LR-41624_normal","327907-LR-41624_tumor"]
normal=["327905-LR-41624_normal"]
num_tumor=["327907-LR-41624_tumor"]

path="/path/to/Snakemake/"
genome="/icgc/dkfzlsdf/analysis/B210/references_genome/Mus_musculus.GRCm38.dna_rm.toplevel.fa"

rule all:
    input:  
        "/path/to/Snakemake/AS-327905-LR-41624_normal_R1_val_1.fq",
        "/path/to/Snakemake/AS-327905-LR-41624_normal_R2_val_2.fq",
        "/path/to/Snakemake/AS-327907-LR-41624_tumor_R1_val_1.fq",
        "/path/to/Snakemake/AS-327907-LR-41624_tumor_R2_val_2.fq",
        "/path/to/Snakemake/AS-327905-LR-41624_normal_no_dupl_sort.bam.bai",
        "/path/to/Snakemake/AS-327907-LR-41624_tumor_no_dupl_sort.bam.bai",
        "/path/to/Snakemake/AS-327905-LR-41624_normal_RG.bam"
        "/path/to/Snakemake/AS-327907-LR-41624_tumor_RG.bam"


rule trim_galore:
    input:
        r1="/path/to/Snakemake/AS-{num}_R1.fastq",
        r2="/path/to/Snakemake/AS-{num}_R2.fastq"
    output:
        "/path/to/Snakemake/AS-{num }_R1_val_1.fq",
        "/path/to/Snakemake/AS-{num }_R2_val_2.fq"
    shell:
        "module load trim-galore/0.5.0 ; module load pypy/2.7-6.0.0 ; trim_galore  --output_dir /path/to/Snakemake/  --paired {input.r1} {input.r2}  "  

rule bwa_mem:
    input:
        R1="/path/to/Snakemake/AS-{num}_R1_val_1.fq",
        R2="/path/to/Snakemake/AS-{num}_R2_val_2.fq"
    output:
        "/path/to/Snakemake/AS-{num}.bam"
    shell:
        "module load samtools/default ; module load bwa/0.7.8 ; bwa mem  {genome}  {input.R1} {input.R2} | samtools view -h -b  > {output} "

rule samtools_sort:
    input:
        "/path/to/Snakemake/AS-{num}.bam"
    output:
        "/path/to/Snakemake/AS-{num}_sort.bam"
    shell:
        "module load samtools/default ; samtools sort -n  -O BAM {input} > {output} "

rule remove_duplicates:
    input:
        "/path/to/Snakemake/AS-{num}_sort.bam"
    output:
        outbam="/path/to/Snakemake/AS-{num}_no_dupl_sort.bam",
        metrics="/path/to/Snakemake/AS-{num}_dupl_metrics.txt"
    shell:
        "module load gatk/4.0.9.0 ; gatk MarkDuplicates -I {input}  -O {output.outbam} -M {output.metrics}  --REMOVE_DUPLICATES=true "

rule samtools_index:
    input:
        "/path/to/Snakemake/AS-{num}_no_dupl_sort.bam"
    output:
        "/path/to/Snakemake/AS-{num}_no_dupl_sort.bam.bai"
    shell:
        "module load samtools/default ; samtools index  {input} "

rule add_readgroup_normal:
    input:
        "/path/to/Snakemake/AS-{normal}_no_dupl_sort.bam"
    output:
        "/path/to/Snakemake/AS-{normal}_RG.bam"
    shell:
        "module load gatk/4.0.9.0 ;  gatk AddOrReplaceReadGroups   -PL Illumina -LB  { wildcards.normal }   -PU  { wildcards.normal }  -SM  NORMAL  -I  { input }    -O  {output} "

rule add_readgroup_tumor:
    input:
        "/path/to/Snakemake/AS-{num}_no_dupl_sort.bam"
    output:
        "/path/to/Snakemake/AS-{num_,'.*tumor.*'}_RG.bam"
    shell:
        "module load gatk/4.0.9.0 ;  gatk AddOrReplaceReadGroups   -PL Illumina -LB  { wildcards.num }   -PU  { wildcards.num }  -SM  TUMOR     -I  { input }    -O  {output} "

错误：

Building DAG of jobs...
MissingInputException in line 37 of /home/l136n/Snakefile_mapping_snv_call_pipeline2b1:
Missing input files for rule trim_galore:
/path/to/Luca/Snakemake/AS-327905-LR-41624_normal_RG.bam/path/to/Luca/Snakemake/AS-327907-LR-41624_tumor_RG_R1.fastq
/path/to/Snakemake/AS-327905-LR-41624_normal_RG.bam/path/to/Luca/Snakemake/AS-327907-LR-41624_tumor_RG_R2.fastq

Answer 1

通配符可以在shell中使用语法{wilcards.var}而非{var}进行访问。您在rule add_readgroup_normal中有后者。 Source。

Answer 2

我以为我会提供解决方案，即使该帖子现在有点老了。该错误仅是由于“ {wildcards.var}”内部存在空格。

Snakemake空运行模式下的NameError

2 个答案: