如何从大文件中检索字符串

时间:2014-02-06 06:57:16

标签: python

我编写了一个代码,其中“IDS.txt”是一个制表符分隔文本文件,其中包含下面给出的ID,其中第一列表示ID第二个起始索引和第三列结束索引。

IDs.txt-------

“complete.txt”

我编写的脚本根据“IDs.txt”检索字符串片段NOT正在工作请帮助我修改代码

with open("\Users\Zebrafish\Desktop\IDs.txt") as f: # will get input from the text
    for line in f:
        c = line.split("\t")                  
        for i, x in enumerate(c):                #passing values to  start and end variables 
            if i == 1:
               start = x
            elif i == 2:
                end =  x
            elif i == 0:
                 gene_name = x
        infile = open("/Users/Zebrafish/Desktop/complete.txt")  #file to get large string data 
        for seq in infile:
            seqnew = seq.split("\t")                       # get data as single line 
        retrived = seqnew[int(start):int(end)]             #retrieve  fragment 
        print retrived

4 个答案:

答案 0 :(得分:1)

我不知道您为什么要在\t文件中分割complete.txt,这是您的代码优化:

ids = {}
with open('/Users/Zebrafish/Desktop/ASHISH/IDs.txt') as f:
    for line in f:
       if len(line.strip()):
           # This makes sure you skip blank lines
           id,start,end = line.split('\t')
           ids[id] = (int(start),int(end))

# Here, I assume your `complete.txt` is a file with one long line.
with open('/Users/Zebrafish/Desktop/ASHISH/complete.txt') as f:
    sequence = f.readline()

# For each id, fetch the sequence "chunk:
for id,value in ids.iteritems():
    start, end = value
    print('{} {}'.format(id,sequence[start-1:end]))

答案 1 :(得分:1)

3MB并不大(在可以运行Windows的计算机上)。只需将第二个文件作为单个字符串加载到内存中,即可获取片段:

# populate `id -> (start, end)` map
ids = {} 
with open(r"\Users\Zebrafish\Desktop\ASHISH\IDs.txt") as id_file:
    for line in id_file:
        if line.strip(): # non-blank line
           id, start, end = line.split() 
           ids[id] = int(start), int(end)

# load the file as a single string (ignoring whitespace)
with open("/Users/Zebrafish/Desktop/ASHISH/complete.txt") as seq_file:
    s = "".join(seq_file.read().split()) # or re.sub("\s+", "", seq_file.read())

# print fragments
for id, (start, end) in ids.items():
    print("{id} -> {fragment}".format(id=id, fragment=s[start:end]))

如果complete.txt文件不适合内存;您可以使用mmap以字节序列的形式访问其内容,而无需将整个文件加载到内存中:

from mmap import ACCESS_READ, mmap    

with open("complete.txt") as f, mmap(f.fileno(), 0, access=ACCESS_READ) as s:
    # use `s` here (assume that indices refer to the raw file in this case)
    # e.g., `fragment = s[start:end]`

答案 2 :(得分:0)

删除该行:

seqnew = seq.split("\t") 

只是做:

retrieved = seqnew[int(start):int(end)]

将获得您想要的子字符串。

然后你就可以:

print retrieved

答案 3 :(得分:0)

谨防\t

中的领先IDs.txt
>>> print "\ta\tb\tc"
        a       b       c
>>> "\ta\tb\tc".split("\t")
['', 'a', 'b', 'c']

i == 0是指空文本而不是基因ID。