Question

我有大约50个制表符分隔的文件，我想将第7列打印到新文件。所有文件具有相同数量的列和相同数量的行。在输出中，不同文件中的列应相互贴在一起，由选项卡分隔。

我在考虑使用'ls'，'xargs'和'awk'的组合。所以我找到我想要的所有文件，然后awk打印第7列并创建output.txt

ls /folder/*_name.txt | awk '{print $7}' xargs {} > output.txt

我的主要问题是使用xargs以及如何在输出文件中的不同列中打印所有$ 7

Answer 1

如果我理解你正在尝试做什么，那么使用awk你可以使用

awk -F '\t' 'FNR == 1 { ++file } { col[FNR, file] = $7 } END { for(i = 1; i <= FNR; ++i) { line = col[i, 1]; for(j = 2; j <= file; ++j) { line = line "\t" col[i, j] }; print line } }' file1 file2 file3 file4

代码是

FNR == 1 { ++file }                 # in the first line of a file, increase
                                    # the file counter, so file is the number
                                    # of the file we're processing
{                         
  col[FNR, file] = $7               # remember the 7th column from all lines
}                                   # by line and file number

END {                               # at the end:
  for(i = 1; i <= FNR; ++i) {       # walk through the lines,
    line = col[i, 1]                # paste together the columns in that line
    for(j = 2; j <= file; ++j) {    # from each file
      line = line "\t" col[i, j]
    }
    print line                      # and print the result.
  }
}

编辑：调整以便在运行中而不是在结束时组装线，这可以缩短为

awk -F '\t' 'FNR == 1 && FNR != NR { sep = "\t" } { line[FNR] = line[FNR] sep $7 } END { for(i = 1; i <= FNR; ++i) { print line[i] } }'

那是

FNR == 1 && FNR != NR {   # in the first line, but not in the first file
  sep = "\t"              # set the separator to a tab (in the first it's empty)
}
{                         # assemble the line on the fly
  line[FNR] = line[FNR] sep $7
}
END {                     # and in the end, print the lines.
  for(i = 1; i <= FNR; ++i) {
    print line[i]
  }
}

将自己诅咒，这可以进一步缩短为

awk -F '\t' '{ line[FNR] = line[FNR] sep $7 } ENDFILE { sep = "\t" } END { for(i = 1; i <= FNR; ++i) { print line[i] } }'

...但是其他awk实现（例如mawk）不知道ENDFILE，所以您可能更愿意避免使用它。

Answer 2

我知道这不是很好但你可以使用Python很容易地完成这个。我在5分钟内写了这个并在三个具有相同列和行的文件上进行了测试，并且它可以工作

import csv, os

def getData(fileDir, newFile, COLUMN):
    COLUMN = COLUMN - 1
    newFile = os.path.join(fileDir,newFile)

    #gets all filepaths for all your files in a directory
    filePaths = []
    for file in os.listdir(fileDir):
        filePaths.append(os.path.join(fileDir,file))
    originalData = []
    for f in filePaths:
        file = []
        with open(f, 'rb') as d:
            reader = csv.reader(d, delimiter='\t')
            #header = (reader.next())[COLUMN] #if you have a header in your csv file uncomment this line so it skips it
            for row in reader:
                file.append(row[COLUMN])
        originalData.append(file)

    #gets a count of how many rows are in your file
    rows = len(originalData[0])

    #creates a new list from the old list and it is now structured like below
    #new list = [[File1_Col7_Row1, File2_Col7_Row1, File3_Col7_Row1],[File1_Col7_Row2, File2_Col7_Row2, File3_Col7_Row2]]
    newData = []
    for i in range(rows):
        r = []
        for item in originalData:
            row = item[i]
            r.append(row)
        newData.append(r)

    #writes the new data to a new file
    with open(newFile, 'wb') as f:
        writer = csv.writer(f, delimiter='\t')
        for row in newData:
            writer.writerow(row)




if __name__ == "__main__":      
    #dir where ONLY the tab files reside
    fileDir = "C:\\TabFiles"
    #new file name, it will be dumped in the dir where the other files reside
    newFile = 'newTabFile.txt'
    # the column you want to grab
    columnNum = 7

    getData(fileDir, newFile, columnNum)

Answer 3

我用Python创建了10个文件：

for i in range(1,10):
    fn='file'+str(i)+'.tsv'
    with open(fn, 'w') as f:
        for line in range(1,4):
            f.write('\t'.join('{}, line: {}, col: {}'.format(fn, line, col) for col in range(1,10)))
            f.write('\n')

创建此类型的10个文件：

file1.tsv, line: 1, col: 1  file1.tsv, line: 1, col: 2  file1.tsv, line: 1, col: 3  file1.tsv, line: 1, col: 4  file1.tsv, line: 1, col: 5  file1.tsv, line: 1, col: 6  file1.tsv, line: 1, col: 7  file1.tsv, line: 1, col: 8  file1.tsv, line: 1, col: 9
file1.tsv, line: 2, col: 1  file1.tsv, line: 2, col: 2  file1.tsv, line: 2, col: 3  file1.tsv, line: 2, col: 4  file1.tsv, line: 2, col: 5  file1.tsv, line: 2, col: 6  file1.tsv, line: 2, col: 7  file1.tsv, line: 2, col: 8  file1.tsv, line: 2, col: 9
file1.tsv, line: 3, col: 1  file1.tsv, line: 3, col: 2  file1.tsv, line: 3, col: 3  file1.tsv, line: 3, col: 4  file1.tsv, line: 3, col: 5  file1.tsv, line: 3, col: 6  file1.tsv, line: 3, col: 7  file1.tsv, line: 3, col: 8  file1.tsv, line: 3, col: 9
...
file9.tsv, line: 1, col: 1  file9.tsv, line: 1, col: 2  file9.tsv, line: 1, col: 3  file9.tsv, line: 1, col: 4  file9.tsv, line: 1, col: 5  file9.tsv, line: 1, col: 6  file9.tsv, line: 1, col: 7  file9.tsv, line: 1, col: 8  file9.tsv, line: 1, col: 9
file9.tsv, line: 2, col: 1  file9.tsv, line: 2, col: 2  file9.tsv, line: 2, col: 3  file9.tsv, line: 2, col: 4  file9.tsv, line: 2, col: 5  file9.tsv, line: 2, col: 6  file9.tsv, line: 2, col: 7  file9.tsv, line: 2, col: 8  file9.tsv, line: 2, col: 9
file9.tsv, line: 3, col: 1  file9.tsv, line: 3, col: 2  file9.tsv, line: 3, col: 3  file9.tsv, line: 3, col: 4  file9.tsv, line: 3, col: 5  file9.tsv, line: 3, col: 6  file9.tsv, line: 3, col: 7  file9.tsv, line: 3, col: 8  file9.tsv, line: 3, col: 9

现在您已拥有这些示例文件（这就是答案），只需使用cut：

$ cut -f 7 *.tsv
file1.tsv, line: 1, col: 7
file1.tsv, line: 2, col: 7
file1.tsv, line: 3, col: 7
file2.tsv, line: 1, col: 7
file2.tsv, line: 2, col: 7
file2.tsv, line: 3, col: 7
file3.tsv, line: 1, col: 7
file3.tsv, line: 2, col: 7
file3.tsv, line: 3, col: 7
file4.tsv, line: 1, col: 7
file4.tsv, line: 2, col: 7
file4.tsv, line: 3, col: 7
file5.tsv, line: 1, col: 7
file5.tsv, line: 2, col: 7
file5.tsv, line: 3, col: 7
file6.tsv, line: 1, col: 7
file6.tsv, line: 2, col: 7
file6.tsv, line: 3, col: 7
file7.tsv, line: 1, col: 7
file7.tsv, line: 2, col: 7
file7.tsv, line: 3, col: 7
file8.tsv, line: 1, col: 7
file8.tsv, line: 2, col: 7
file8.tsv, line: 3, col: 7
file9.tsv, line: 1, col: 7
file9.tsv, line: 2, col: 7
file9.tsv, line: 3, col: 7

然后使用tr隐藏这些结果：

$ cut -f 7 *.tsv | tr '\n' '\t'
file1.tsv, line: 1, col: 7  file1.tsv, line: 2, col: 7  file1.tsv, line: 3, col: 7  file2.tsv, line: 1, col: 7  file2.tsv, line: 2, col: 7  file2.tsv, line: 3, col: 7  file3.tsv, line: 1, col: 7  file3.tsv, line: 2, col: 7  file3.tsv, line: 3, col: 7  file4.tsv, line: 1, col: 7  file4.tsv, line: 2, col: 7  file4.tsv, line: 3, col: 7  file5.tsv, line: 1, col: 7  file5.tsv, line: 2, col: 7  file5.tsv, line: 3, col: 7  file6.tsv, line: 1, col: 7  file6.tsv, line: 2, col: 7  file6.tsv, line: 3, col: 7  file7.tsv, line: 1, col: 7  file7.tsv, line: 2, col: 7  file7.tsv, line: 3, col: 7  file8.tsv, line: 1, col: 7  file8.tsv, line: 2, col: 7  file8.tsv, line: 3, col: 7  file9.tsv, line: 1, col: 7  file9.tsv, line: 2, col: 7  file9.tsv, line: 3, col: 7

或paste：

$ cut -f 7 *.tsv | paste -s -d '\t' - 
file1.tsv, line: 1, col: 7  file1.tsv, line: 2, col: 7  file1.tsv, line: 3, col: 7  file2.tsv, line: 1, col: 7  file2.tsv, line: 2, col: 7  file2.tsv, line: 3, col: 7  file3.tsv, line: 1, col: 7  file3.tsv, line: 2, col: 7  file3.tsv, line: 3, col: 7  file4.tsv, line: 1, col: 7  file4.tsv, line: 2, col: 7  file4.tsv, line: 3, col: 7  file5.tsv, line: 1, col: 7  file5.tsv, line: 2, col: 7  file5.tsv, line: 3, col: 7  file6.tsv, line: 1, col: 7  file6.tsv, line: 2, col: 7  file6.tsv, line: 3, col: 7  file7.tsv, line: 1, col: 7  file7.tsv, line: 2, col: 7  file7.tsv, line: 3, col: 7  file8.tsv, line: 1, col: 7  file8.tsv, line: 2, col: 7  file8.tsv, line: 3, col: 7  file9.tsv, line: 1, col: 7  file9.tsv, line: 2, col: 7  file9.tsv, line: 3, col: 7

将多个文件中的相同列粘贴到一个文件中

3 个答案: