awk匹配基于行的文件的模式并输出为CSV

时间:2017-09-02 03:30:30

标签: regex awk gawk

我有一个包含此类格式记录的文件:

LOCUS       NG_029783              19834 bp    DNA     linear   PRI 03-OCT-2014 DEFINITION  Homo sapiens long intergenic non-protein coding RNA 1546
            (LINC01546), RefSeqGene on chromosome X. ACCESSION   NG_029783 VERSION     NG_029783.1 KEYWORDS    RefSeq; RefSeqGene. SOURCE      Homo sapiens (human)   ORGANISM  Homo sapiens
            Eukaryota; Metazoa; Chordata; Craniata; Vertebrata; Euteleostomi;
            Mammalia; Eutheria; Euarchontoglires; Primates; Haplorrhini;
            Catarrhini; Hominidae; Homo. COMMENT     VALIDATED REFSEQ: This record has undergone validation or
            preliminary review. The reference sequence was derived from
            AC004616.1.
            This sequence is a reference standard in the RefSeqGene project. PRIMARY     REFSEQ_SPAN         PRIMARY_IDENTIFIER PRIMARY_SPAN        COMP
            1-19834             AC004616.1         8636-28469 FEATURES             Location/Qualifiers
     source          1..19834
                     /organism="Homo sapiens"
                     /mol_type="genomic DNA"
                     /db_xref="taxon:9606"
                     /chromosome="X"
                     /map="Xp22.33"
     variation       4
                     /replace="c"
                     /replace="t"
                     /db_xref="dbSNP:1205550"
     variation       17
                     /replace="c"
                     /replace="t"
                     /db_xref="dbSNP:1205551"
     gene            5001..5948
                     /gene="OR6K3"
                     /gene_synonym="OR1-18"
                     /note="olfactory receptor family 6 subfamily K member 3"
                     /db_xref="GeneID:391114"
                     /db_xref="HGNC:HGNC:15030"
     mRNA            5001..5948
                     /gene="OR6K3"
                     /gene_synonym="OR1-18"
                     /product="olfactory receptor family 6 subfamily K member

// 
LOCUS       NG_032962              70171 bp    DNA     linear   PRI 17-JUN-2016 DEFINITION  Homo sapiens death domain containing 1 (DTHD1), RefSeqGene on
                chromosome 4. ACCESSION   NG_032962 VERSION     NG_032962.1 KEYWORDS    RefSeq; RefSeqGene. SOURCE      Homo sapiens (human)   ORGANISM  Homo sapiens
                Eukaryota; Metazoa; Chordata; Craniata; Vertebrata; Euteleostomi;
                Mammalia; Eutheria; Euarchontoglires; Primates; Haplorrhini;
                Catarrhini; Hominidae; Homo. COMMENT     REVIEWED REFSEQ: This record has been curated by NCBI staff. The
                reference sequence was derived from AC104078.3.
                This sequence is a reference standard in the RefSeqGene project.

        Summary: This gene encodes a protein which contains a death domain.
        Death domain-containing proteins function in signaling pathways and
        formation of signaling complexes, as well as the apoptosis pathway.
        Alternative splicing results in multiple transcript variants.
        [provided by RefSeq, Oct 2012]. PRIMARY     REFSEQ_SPAN         PRIMARY_IDENTIFIER PRIMARY_SPAN        COMP
        1-70171             AC104078.3         59395-129565 FEATURES             Location/Qualifiers
 source          1..70171
                 /organism="Homo sapiens"
                 /mol_type="genomic DNA"
                 /db_xref="taxon:9606"
                 /chromosome="4"
                 /map="4p14"
 gene            5001..129091
                 /gene="REEP1"
                 /gene_synonym="C2orf23; HMN5B; SPG31; Yip2a"
                 /note="receptor accessory protein 1"
                 /db_xref="GeneID:65055"
                 /db_xref="HGNC:HGNC:25786"
                 /db_xref="MIM:609139"
 mRNA            join(5001..5060,60842..60914,79043..79119,88270..88390,
                 91014..91127,110282..110459,125974..129091)
                 /gene="REEP1"
                 /gene_synonym="C2orf23; HMN5B; SPG31; Yip2a"
                 /product="receptor accessory protein 1, transcript variant
                 1"
                 /transcript_id="NM_001164730.1"

我一直在使用这个工作流程:

  1. 删除空格 gawk '{$1=$1}1' raw_file.text > temp_file.txt

  2. 匹配"摘要"内容gawk /Summary/,/\]/{print} temp_file.text > summary_temp.txt

  3. 删除换行gawk 'BEGIN {RS=""}{gsub(/\n/,"",$0); print $0}' summary_temp.text > summary.txt

  4. 我有几个问题。 首先,我如何将这3个步骤结合起来。 其次,我怎么能选择一个或多个额外的匹配,例如匹配' / gene =" AP3B2"' (这需要匹配" / gene"" gene"之后的第一个实例),以便我可以输出以下形式的内容:

    基因,摘要

1 个答案:

答案 0 :(得分:2)

$ cat tst.awk
BEGIN{RS="//"}
{
  match($0, /\/gene="([^"]+)"/, a)
  print a[1] ", ", 
        gensub(/\s\s+/, "", "g", gensub(/.*Summary:\s([^\[]+).*/, "\\1", "g"))
}

<强>解释

match($0, /\/gene="([^"]+)"/, a)

抓住所有&#34; \ gene&#34;数组中的部分a。根据你的问题,只需要第一次出现,即[1](而不是AP3B2,顺便说一句)。

gensub(/.*Summary:\s([^\[]+).*/, "\\1", "g")

在&#34之后抓住所有内容;摘要:&#34;直到找到一个&#34; [&#34;。 最后一个结果有空格和换行符。让我们摆脱它们:

gensub(/\s\s+/, "", "g", <<result of 1st gensub>>)
  

编辑:并非每条记录都包含&#34;摘要&#34;

将脚本更改为:

$ cat tst.awk
BEGIN{RS="//"}
{
  match($0, /\/gene="([^"]+)"/, a)
  match($0, /Summary:\s([^\[]+)/, b)
  print a[1] ",",
    gensub(/\s\s+/, " ", "g", b[1])
}

使用OP提供的输入运行脚本:

awk -f tst.awk tst.txt
OR6K3,
REEP1,  This gene encodes a protein which contains a death domain.Death 
domain-containing proteins function in signaling pathways andformation 
of signaling complexes, as well as the apoptosis pathway.Alternative 
splicing results in multiple transcript variants.