Question

我在CSV目录中有两个/tmp/文件。

一个CSV文件的结果来自python个结果，第二个CSV文件是要匹配的主文件。

>>> import json
>>> resp = { "status":"success", "msg":"", "data":[ { "website":"https://www.blahblah.com", "severity":"low", "location":"unknown", "asn_number":"AS4134 Chinanet", "longitude":121.3997000000, "epoch_timestamp":1530868957, "id":"c1e15eccdd1f31395506fb85" }, { "website":"https://www.jhonedoe.co.uk/sample.pdf", "severity":"low", "location":"unknown", "asn_number":"AS4134 Chinanet", "longitude":120.1613998413, "epoch_timestamp":1530868957, "id":"933bf229e3e95a78d38223b2" } ] }
>>> response = json.loads(json.dumps(resp))
>>> KEYS = 'website', 'asn_number' , 'severity'
>>> x = []
>>> for attribute in response['data']:
...     csv_response = ','.join(attribute[key] for key in KEYS)
...     with open('/tmp/processed_results.csv', 'a') as score:
...             score.write(csv_response + '\n')

$cat processed_results.csv

https://www.blahblah.com,AS4134 Chinanet,low
https://www.jhonedoe.co.uk/sample.pdf,AS4134 Chinanet,low

要匹配的元文件。

$cat master_meta.csv
http://download2.freefiles-10.de,AS24940 Hetzner Online GmbH,high
https://www.jhonedoe.co.uk/sample.pdf,AS4134 Chinanet,low
http://download2.freefiles-11.de,AS24940 Hetzner Online GmbH,high
www.solener.com,AS20718 ARSYS INTERNET S.L.,low
https://www.blahblah.com,AS4134 Chinanet,low
www.telewizjairadio.pl,AS29522 Krakowskie e-Centrum Informatyczne JUMP Dziedzic,high

我知道如何使用grep比较两个文件并获得匹配的行。

$grep -Ff processed_results.csv master_meta.csv

https://www.jhonedoe.co.uk/sample.pdf,AS4134 Chinanet,low
https://www.blahblah.com,AS4134 Chinanet,low

关于如何使用python subprocess call传递grep / sed / awk命令来比较两个文件并获取匹配行的任何建议可变的？

Answer 1

大多数人不会称其为“好”，但是如果您只是自己使用它，则不必太在意。

def sh(cmd, verbose=True):
    """Returns the stdout of the shell command as an iterator over the lines,
    but the process self is blocking. 
    The lines are strip()-ed.
    """
    if verbose:
        print("[INFO]: executing: " + cmd)
    out, err = Popen(cmd, stdout=PIPE, stderr=PIPE, shell=True).communicate()
    if err:
        print("[ERROR]: while executing: " + cmd)
        print(err.decode('ascii').strip())
    return out.decode('ascii').strip()

python子进程调用，以使用grep或awk或sed比较两个CSV文件

1 个答案: