Question

我在bash脚本中使用grep，然后在pandas中读取为csv

因为我的日志文件如下所示

somegarbage propercsv somegarbage propercsv

我grep propercsv，生成一个csv文件，然后使用下面的代码加载它。我想只在python中嵌入它。这是从作为参数

给出的日志文件中过滤掉csv

#!/usr/bin/python
import csv
import sys

with open(sys.argv[1], 'rb') as f_input, open(sys.argv[2], 'wb') as f_output:
    csv_output = csv.writer(f_output)

    csv_output.writerow(cols)

    for row in csv.reader(f_input, delimiter='|'):
        # Remove any entries that do not have a colon
        row = [c for c in row if c.find(':') != -1]
        # Convert remaining columns into a dictionary
        entries = {c.split(':')[0].strip() : c.split(':')[1].strip() for c in row}
        csv_output.writerow([entries.get(c, "") for c in cols])

Answer 1

您可以简单地过滤日志文件中导致适当数量的csv列（例如＆gt; = 3）的行，如下所示：

import pandas as pd
from itertools import ifilter
import csv

data = []
cols = ['Name', 'Place', 'Time']

with open('logfile.txt', 'rb') as f_input:
    for row in ifilter(lambda x: len(x) >= 3, csv.reader(f_input, delimiter='|')):
        row = [c for c in row if c.find(':') != -1]
        entries = {c.split(':')[0].strip() : c.split(':')[1].strip() for c in row}
        data.append(entries.get(c, "") for c in cols)

print pd.DataFrame(data, columns=cols)

给你以下输出：

  Name Place Time
0  abc   Xyz  123
1  abc   Xyz  123
2  abc   Xyz  123
3  abc   Xyz  123

过滤文件内容然后在python中读取为csv

1 个答案: