从大文件中的模式的特定出现中提取名称

时间:2017-11-15 15:10:02

标签: python r bash string-matching

我有一个FASTA文件,它基本上是一个文本文件,用于描述具有超过10,000个FASTA序列的生物序列数据(https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&PAGE_TYPE=BlastDocs&DOC_TYPE=BlastHelp)(以>开头)。文件的开头如下:

>Gene A
GAACTACACAAACGTAAAATGTAAAACAAAGGTATAAATTCCAGAAGTTGGACAGACATATATAGACAGCACATATATTA
TCTTTATTTTTTTATGTATGATAACATTAAATATAACGTTCAACAATT
>Gene B
GAACTACACAAACGTAAAATGTAAAACAAAGGTATAAATTCCAGAAGTTGGACAGACATATATAGACAGCACATATATTA
TCTTTATTTTTTTATGTATGATAACATTAAATATAACGTTCAACAATTACACCGTTAGCAGTGTGAGCAAAAACGATTAA
AAAGTAAATATTATAAAAGCCCTC
>Gene C
AACAACAAATTGCCATCTACCCGTTTGAATCCTGTAATAATAACTTGCCCAGATTTGCTGCAGCATACTCCTAGAGTTGG
GCTGGGTGGCCCACACAAGCGATAATAACATTTAACAATTGTTTGATATATGTACTTTTTTTTAAGTTTTTTTCTCCTCG
TACTTGCCTTCCAAAAACTCGTTAGCTTTGTACACATACGCCTTTAATTAAAATACTGATAGATGCGTACCACTTACGTC
ATTAGAAAAAGTCACCAAAAGGAAAAATATGGACGACACAAGAACGAGGAGATCTAAGCCACTCGTAGACCACTAAGCAC
AAAATACCCGAAAAATATAACTGATATGATTGCCAACTACCCTGCGACTATGTAAACCCAACCTTCCCCCCTCCTTTACC
CTCTTATTCAAATCGACGCGTGTGTAGAAGATACACTTATTATATTTTTTTTCTGAGATACAATTATAAACACAAAAACG
ACTTTTAACTATATATTAAATAAAAACAAAAGGAAAAACATAATAATTT
>Gene D
AACAACAAATTGCCATCTACCCGTTTGAATCCTGTAATAATAACTTGCCCAGATTTGCTGCAGCATACTCCTAGAGTTGG
GCTGGGTGGCCCACACAAGCGATAATAACATTTAACAATTGTTTGATATATGTACTTTTTTTTAAGTTTTTTTCTCCTCG
TACTTGCCTTCCAAAAACTCGTTAGCTTTGTACACATACGCCTTTAATTAAAATACTGATAGATGCGTACCACTTACGTC
ATTAGAAAAAGTCACCAAAAGGAAAAATATGGACGACACAAGAACGAGGAGATCTAAGCCACTCGTAGACCACTAAGCAC
AAAATACCCGAAAAATATAACTGATATGATTGCCAACTACCCTGCGACTATGTAAACCCAACCTTCCCCCCTCCTTTACC
CTCTTATTCAAATCGACGCGTGTGTAGAAGATACACTTATTATATTTTTTTTCTGAGATACAATTATAAACACAAAAACG
ACTTTTAACTATATATTAAATAAAAACAAAAGGAAAAACATAATAATTT

等约10,000个基因。 我想:

  1. 找出哪些基因含有特定模式(CTTTGTA)
  2. 该基因中存在多少次这种模式?
  3. 导出包含具有模式频率的模式的基因名称列表。
  4. 赞赏Bash或Python(或R)中的任何解决方案。

    P.S。到目前为止我尝试了但是没有用到:将带有序列的基因提取到单独的文件中,然后在单独的文件中grep查看模式。但是,我可以不生成这些单独的文件。我用了

    grep '^>' file.txt > new_file.txt
    

    但我得到的输出是一个只包含所有基因名称的文件。

2 个答案:

答案 0 :(得分:0)

sequences = open('fastafile.txt').read().split('>')  # Creates a list of sequences.

needle = 'CTTTGTA'

occurrences = {}

for sequence in sequences:
   occ = sequence.count(needle)  # Returns the number of times the substring occurs in the string sequence.
   if occ:  # If greater than 0, create an entry in our dictionary. The sequence being the key and the count the value.
      occurrences[sequence] = occ

output = []

sorted_occurrences = sorted(occurrences.items(), key=operator.itemgetter(1))  # Sort the dictionary by length, so sequences with the highest occurrence of the needle appear at the top.

for seq, occ_count in sorted_occurrences.iteritems():
    gene_name, sequence = seq.split('\n')
    formatted_line = '{gene_name} - {occ_count}'.format(gene_name=gene_name, occ_count=str(occ_count))  # Format the lines the way you want.
    output.append(formatted_line)  

with open('occurences.txt') as o_f:
    o_f.write('\n'.join(output))

答案 1 :(得分:0)

这是R使用stringi包的解决方案。由于没有单个文本文件或类似文件作为可重现的示例进行访问,因此我使用了使用cat()readlines()来读取表示您提供的行副本的临时文件。还请检查时间基准,可能对大文件感兴趣。

library(stringi)

cat(">Gene A
GAACTACACAAACGTAAAATGTAAAACAAAGGTATAAATTCCAGAAGTTGGACAGACATATATAGACAGCACATATATTA
    TCTTTATTTTTTTATGTATGATAACATTAAATATAACGTTCAACAATT
    >Gene B
    GAACTACACAAACGTAAAATGTAAAACAAAGGTATAAATTCCAGAAGTTGGACAGACATATATAGACAGCACATATATTA
    TCTTTATTTTTTTATGTATGATAACATTAAATATAACGTTCAACAATTACACCGTTAGCAGTGTGAGCAAAAACGATTAA
    AAAGTAAATATTATAAAAGCCCTC
    >Gene C
    AACAACAAATTGCCATCTACCCGTTTGAATCCTGTAATAATAACTTGCCCAGATTTGCTGCAGCATACTCCTAGAGTTGG
    GCTGGGTGGCCCACACAAGCGATAATAACATTTAACAATTGTTTGATATATGTACTTTTTTTTAAGTTTTTTTCTCCTCG
    TACTTGCCTTCCAAAAACTCGTTAGCTTTGTACACATACGCCTTTAATTAAAATACTGATAGATGCGTACCACTTACGTC
    ATTAGAAAAAGTCACCAAAAGGAAAAATATGGACGACACAAGAACGAGGAGATCTAAGCCACTCGTAGACCACTAAGCAC
    AAAATACCCGAAAAATATAACTGATATGATTGCCAACTACCCTGCGACTATGTAAACCCAACCTTCCCCCCTCCTTTACC
    CTCTTATTCAAATCGACGCGTGTGTAGAAGATACACTTATTATATTTTTTTTCTGAGATACAATTATAAACACAAAAACG
    ACTTTTAACTATATATTAAATAAAAACAAAAGGAAAAACATAATAATTT
    >Gene D
    AACAACAAATTGCCATCTACCCGTTTGAATCCTGTAATAATAACTTGCCCAGATTTGCTGCAGCATACTCCTAGAGTTGG
    GCTGGGTGGCCCACACAAGCGATAATAACATTTAACAATTGTTTGATATATGTACTTTTTTTTAAGTTTTTTTCTCCTCG
    TACTTGCCTTCCAAAAACTCGTTAGCTTTGTACACATACGCCTTTAATTAAAATACTGATAGATGCGTACCACTTACGTC
    ATTAGAAAAAGTCACCAAAAGGAAAAATATGGACGACACAAGAACGAGGAGATCTAAGCCACTCGTAGACCACTAAGCAC
    AAAATACCCGAAAAATATAACTGATATGATTGCCAACTACCCTGCGACTATGTAAACCCAACCTTCCCCCCTCCTTTACC
    CTCTTATTCAAATCGACGCGTGTGTAGAAGATACACTTATTATATTTTTTTTCTGAGATACAATTATAAACACAAAAACG
    ACTTTTAACTATATATTAAATAAAAACAAAAGGAAAAACATAATAATTT
", file = "tempfile.txt")

genes <- readLines("tempfile.txt", n=-1)
unlink("tempfile.txt")

genes <- unlist(stri_split_fixed(paste(genes, collapse = " "), ">"))
genes <- genes[ genes != ""]

genenames <- unlist(stri_extract_all_regex(genes, "Gene \\w+"))
genes <- stri_replace_all_fixed(genes, genenames, "")
names(genes) <- genenames

genes <- gsub("\\s+", "", genes, perl = T) 

gene_pattern_freq <- function(str, patterns) {

  res <- sapply(patterns, function(p) {

    stringi::stri_count_fixed(str, p)

  }, USE.NAMES = T)

  rownames(res) <- names(str)

  return(res)
}

searchpatterns <- c("AA", "GT", "GAACTACACAAACGTAAAATGTAAAACAAAGGTATAAA")
result <- gene_pattern_freq(genes, searchpatterns)
result
#        AA GT GAACTACACAAACGTAAAATGTAAAACAAAGGTATAAA
# Gene A 14  6                                      1
# Gene B 21 10                                      1
# Gene C 52 18                                      0
# Gene D 52 18                                      0

library(microbenchmark)
microbenchmark(gene_pattern_freq(genes, searchpatterns))
# Unit: microseconds
# expr                                      min     lq    mean   median   uq     max   neval
# gene_pattern_freq(genes, searchpatterns) 68.687 77.371 123.438 78.161 79.345 4479.19   100

#export
write.csv(result, file = "../mypath/gene_pattern_freq_result.csv" )