Question

我期待一种读取日志文件并基于第5列读取主机名的方法，并在所有迭代过程中保留主机名，直到新名称出现，或者说找到新名称时仅标记双倍空格，但需要打印整行。

只需读取文件：

$ cat test.py
with open("file", "r") as f:
    for line in f:
        fr = f.read()
        print(fr)

文件内容输出：

$ ./test.py
Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw070 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:17:26 hw070 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:16:21 hw071 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:17:24 hw071 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:18:27 hw071 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:43:07 hw073 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 15:45:59 hw074 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 17:17:12 hw074 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 01:20:20 hw074 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 16:16:01 hw079 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:17:56 hw079 kernel: nfs: server hpstor001   not responding, still trying
Jul 17 04:34:08 anujv kernel: nfs: server hpstor003  not responding, still trying
Jul 17 21:18:42 sanujv kernel: nfs: server hpstor003  not responding, still trying
Jul 18 01:36:00 sanujv kernel: nfs: server hpstor003  not responding, still trying
Jul 18 04:16:10 sanujv kernel: nfs: server hpstor003  not responding, still trying

所需的输出：

Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying


Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying


Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying

或者再加一点化妆品：

hw067 
Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying

hw068 
Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying

hw069
Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying

我们可以使用Python Pandas做到这一点吗？ -谢谢

Answer 1

您可以通过itertools和operator函数进行操作，我是学习者，我从POST的ans中借用了代码，但是我收集了一些关于这两个函数的详细说明，希望如此。

itertools.groupby（iterable，key =无或某些功能）获取可迭代项的列表，并根据指定的密钥对它们进行分组。键指定对每个可迭代对象要执行的操作，然后将其结果用作每个项分组的标题；最终具有相同“键”值的项目将归入同一组。返回值类似于字典，它的迭代形式是{key：value}。

operator.itemgetter（n）构造一个可调用对象，该对象假定一个可迭代对象（例如list，tuple，set）作为输入，并从中获取第n个元素。

operator.itemgetter（n）将为您提供：一个从类似列表的对象中获取项目的函数。像operator.itemgetter（3）一样，它从列表中获取了第三项。

更确切地说，这些是可调用的，而不是函数，但这是一个通常可以忽略的差异。

以下代码应该对您有用：

import itertools
import operator

log_list = []

file = open('file')
for line in file:
    loggs = line.strip().split()
    log_list.append(loggs)

key = operator.itemgetter(3)
# this step is only required if the list is not sorted by the key
log_list.sort(key=key)
for index, logs in itertools.groupby(log_list, key):
    print(index)
    for log in logs:
        print("    " + " ".join(log))
    print("")

以上代码的结果，我已将您的文件用于测试。

anujv
    Jul 17 04:34:08 anujv kernel: nfs: server hpstor003 not responding, still trying

hw060
    Jul 18 15:46:02 hw060 kernel: nfs: server hpstor001 not responding, still trying

hw067
    Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002 not responding, still trying
    Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001 not responding, still trying
    Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001 not responding, still trying
    Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001 not responding, still trying

hw068
    Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002 not responding, still trying
    Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001 not responding, still trying
    Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001 not responding, still trying

hw069
    Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001 not responding, still trying
    Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001 not responding, still trying
    Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001 not responding, still trying
    Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001 not responding, still trying

hw070
    Jul 18 22:13:20 hw070 kernel: nfs: server hpstor001 not responding, still trying
    Jul 18 22:17:26 hw070 kernel: nfs: server hpstor002 not responding, still trying

hw071
    Jul 18 20:16:21 hw071 kernel: nfs: server hpstor002 not responding, still trying
    Jul 18 20:17:24 hw071 kernel: nfs: server hpstor002 not responding, still trying
    Jul 18 20:18:27 hw071 kernel: nfs: server hpstor002 not responding, still trying

hw073
    Jul 18 20:43:07 hw073 kernel: nfs: server hpstor002 not responding, still trying

hw074
    Jul 18 15:45:59 hw074 kernel: nfs: server hpstor001 not responding, still trying
    Jul 18 17:17:12 hw074 kernel: nfs: server hpstor002 not responding, still trying
    Jul 19 01:20:20 hw074 kernel: nfs: server hpstor002 not responding, still trying

hw079
    Jul 18 16:16:01 hw079 kernel: nfs: server hpstor001 not responding, still trying
    Jul 19 00:17:56 hw079 kernel: nfs: server hpstor001 not responding, still trying

sanujv
    Jul 17 21:18:42 sanujv kernel: nfs: server hpstor003 not responding, still trying
    Jul 18 01:36:00 sanujv kernel: nfs: server hpstor003 not responding, still trying
    Jul 18 04:16:10 sanujv kernel: nfs: server hpstor003 not responding, still trying

python根据主机名从复合日志文件中分隔行

1 个答案: