有效的方法来比较很长的列表之间的元素

时间:2013-01-21 08:31:21

标签: python match performance

我在文件中有两个列表: 文件1有200000行,看起来像

MAP2K4  FLNC
MYPN    ACTN2
ACVR1   FNTA
UGT2A1  HPGDS
RPA2    STAT3
ARF1    GGA3
ARF3    ARFIP2
ARF3    ARFIP1
AKR1A1  EXOSC4
RPA2    GAS7
APP APPBP2
APLP1   DAB1
CITED2  TFAP2A
EP300   TFAP2A
APOB    MTTP
ARRB2   RALGDS
ARRB2   ZNF807

文件2有700000行,看起来像:

MAP2K4  FLNC
MAP2K4  rs10036867
MAP2K4  ACTN2
MAP2K4  TEP1
ACTN2   MYPN
UGT2A1  NDUFAF6
RPA2    rs10109257
RPA2    rs10151961
GAS7    RPA2
APOB    PDZRN4
APOB    BICD1
ARRB2   ZNF807
ARRB2   FAM107B

我需要在这两个列表之间获得匹配的行,尽管元素的顺序。例如,在上面的示例中,它应该如下所示:

MAP2K4  FLNC
ACTN2   MYPN
RPA2    GAS7
ARRB2   ZNF807

我写了以下内容,但这似乎需要永远!

col0_file1 = []
col1_file1 = []
col0_file2 = []
col1_file2 = []
with open('File1') as f1, open('File2') as f2:
    for line in f1:
        col0,col1 = line.split()
        col0_file1.append(col0)
        col1_file1.append(col1)
    for line in f2:
        col0,col1 = line.split()
        col0_file2.append(col0)
        col1_file2.append(col1)

result = []
for x in range(len(col0_file1)):
    for i, j in map(None, col0_file2, col1_file2):
        if i == col0_file1[x] and j == col1_file1[x]:
            result.append([i,j])
        elif j == col0_file1[x] and i == col1_file[x]:
            result.append([i,j])

with open('matching', 'w') as out:
    for elem in result:
        out.write('{a} \n'.format(a = '\t'.join(elem)))

任何方式我都可以简化复杂性?或更好的做法?

1 个答案:

答案 0 :(得分:4)

我说,做两个set并走一条路口:

with open('File1') as f1, open('File2') as f2:
    columns_a = set(tuple(sorted(l.split())) for l in f1)
    columns_b = set(tuple(sorted(l.split())) for l in f2)

with open('matching', 'w') as out:
    for elem in columns_a  & columns_b:
        out.write('{a} \n'.format(a = '\t'.join(elem)))