Python:比较具有相似名称的文件(递归)

时间:2012-05-23 03:39:38

标签: python file-io

these guys的帮助下,我能够生成以下代码,该代码读入两个文件(即SA1.WRD和SA1.PHN),合并它们,并将结果与​​子进行比较 - 从字典中删除的单词列表:

    导入系统     进口口     进口重新     import itertools

#generator function to merge sound and word files
def takeuntil(iterable, stop):
    for x in iterable:
        yield x
        if x[1] == stop:
            break

#open a dictionary file and create subset of words
class_defintion = re.compile('([1-2] [lnr] t en|[1-2] t en)')
with open('TIMITDIC.TXT') as w_list:
    entries = (line.split(' ', 1) for line in w_list)
    comp_set = [ x[0] for x in entries if class_defintion.search(x[1]) ]

#open word and sound files
total_words = 0
with open(sys.argv[1]) as unsplit_words, open(sys.argv[2]) as unsplit_sounds:
    sounds = (line.split() for line in unsplit_sounds)
    words = (line.split() for line in unsplit_words)
    output = [
    (word, " ".join(sound for _, _, sound in
        takeuntil(sounds, stop)))
    for start, stop, word in words
]
for x in output:
    total_words += 1

#extract words from above into list of words in dictionary set
glottal_environments = [ x for x in output if x[0] in comp_set ]

我正在尝试修改#open a dictionary files之后的部分,以便在包含多个子目录的大型目录上运行。每个子目录都包含.txt文件,.wav文件,.wrd和.phn文件。我想只打开.wrd和.phn文件,我希望能够一次打开两个文件,并且只有当基本文件名匹配时,即SA1.WRD和SA1.PHN,而不是SA1。 WRD和SI997.PHN。

我的直接猜测是做这样的事情:

for root, dir, files in os.walk(sys.argv[1]):
    words = [f for f in files if f.endswith('.WRD')]
    phones = [f for f in files if f.endswith('.PHN')]
    phones.sort()
    words.sort()
    files = zip(words, phones)

返回:[('SA1.WRD', 'SA1.PHN'), ('SA2.WRD', 'SA2.PHN'), ('SI997.WRD', 'SI997.PHN')]

我的第一个问题是我是否在正确的轨道上,如果是这样,我的第二个问题是如何将这些元组中的每个项目视为要读取的文件名。

感谢您提供的任何帮助。

修改

我想我可以把代码块放到for循环中:

for f in files:
        #OPEN THE WORD AND PHONE FILES, COMAPRE THEM (TAKE A WORD COUNT)
        total_words = 0
        with open(f[0]) as unsplit_words, open(f[1]) as unsplit_sounds:

        ...

然而,这会导致IOError,可能是由于每个元组中每个项目周围的单引号。

更新 我修改了原始脚本以包含os.path.join(root, f),如下所示。该脚本现在遍历目录树中的所有文件,但它只处理它找到的最后两个文件。以下是print files的输出:

[]
[('test/test1/SI997.WRD', 'test/test1/SI997.PHN')]
[('test/test2/SI997.WRD', 'test/test2/SI997.PHN')]

1 个答案:

答案 0 :(得分:1)

我测试了与文件系统相关的不同部分,但您更容易确认实际文件以确认它对您的数据有效。

编辑以允许包含路径名

import sys
import os
import os.path
import re
import itertools

#generator function to merge sound and word files
def takeuntil(iterable, stop):
    for x in iterable:
        yield x
        if x[1] == stop:
            break

def process_words_and_sounds(word_file, sound_file):
    #open word and sound files
    total_words = 0
    with open(word_file) as unsplit_words, open(sound_file) as unsplit_sounds:
        sounds = (line.split() for line in unsplit_sounds)
        words = (line.split() for line in unsplit_words)
        output = [
            (word, " ".join(sound for _, _, sound in
                            takeuntil(sounds, stop)))
            for start, stop, word in words
            ]
        for x in output:
            total_words += 1
    return total_words, output

for root, dir, files in os.walk(sys.argv[1]):
    words = [ os.path.join( root, f ) for f in files if f.endswith('.WRD')]
    phones = [ os.path.join( root, f ) for f in files if f.endswith('.PHN')]
    phones.sort()
    words.sort()
    files = zip(words, phones)
    # print files

output = []
total_words = 0
for word_sounds in files:
    word_file, sound_file = word_sounds
    word_count, output_subset = process_words_and_sounds(word_file, sound_file)
    total_words += word_count
    output.extend( output_subset )

#open a dictionary file and create subset of words
class_defintion = re.compile('([1-2] [lnr] t en|[1-2] t en)')
with open('TIMITDIC.TXT') as w_list:
    entries = (line.split(' ', 1) for line in w_list)
    comp_set = [ x[0] for x in entries if class_defintion.search(x[1]) ]

#extract words from above into list of words in dictionary set
glottal_environments = [ x for x in output if x[0] in comp_set ]