Python:需要从日志文件中获取唯一错误

时间:2013-07-16 23:18:16

标签: python file split log4j unique

到目前为止我有什么

def unique_ips():
f = open('logfile','r')
ips = set()
for line in f:
    ip = line.split()[0]
    print ip
    for date in ip:
       logdate = line.split()[3]
       print "\t", logdate
       for entry in logdate:
           info = line.split()[5:11] 
           print "\t\t", info
    ips.add(ip)
unique_ips()

我遇到麻烦的部分是:

       for entry in logdate:
           info = line.split()[5:20] 
           print "\t\t", info

我有一个日志文件,我必须首先通过ip排序,然后按时间排序然后错误

应该是这样的:

199.21.99.83
        [30/Jun/2013:07:18:30
                ['"GET', '/searchme/index.php?f=man_soweth', 'HTTP/1.1"', '200', '8676', '"-"']

但我得到了:

199.21.99.83
        [30/Jun/2013:07:18:30
                ['"GET', '/searchme/index.php?f=man_soweth', 'HTTP/1.1"', '200', '8676', '"-"']
                ['"GET', '/searchme/index.php?f=man_soweth', 'HTTP/1.1"', '200', '8676', '"-"']
                ['"GET', '/searchme/index.php?f=man_soweth', 'HTTP/1.1"', '200', '8676', '"-"']
                ['"GET', '/searchme/index.php?f=man_soweth', 'HTTP/1.1"', '200', '8676', '"-"']
                 ...

我确定我遇到了某种语法问题但会很感激帮助!

日志文件如下所示:

99.21.99.83 - - [30/Jun/2013:07:15:50 -0500] "GET /lenny/index.php?f=13 HTTP/1.1" 200 11244 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
199.21.99.83 - - [30/Jun/2013:07:16:13 -0500] "GET /searchme/index.php?f=being_fruitful HTTP/1.1" 200 7526 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
199.21.99.83 - - [30/Jun/2013:07:16:45 -0500] "GET /searchme/index.php?f=comparing_themselves HTTP/1.1" 200 7369 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
66.249.73.40 - - [30/Jun/2013:07:16:56 -0500] "GET /espanol/displayAncient.cgi?ref=isa%2054:3 HTTP/1.1" 500 167 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
199.21.99.83 - - [30/Jun/2013:07:17:00 -0500] "GET /searchme/index.php?f=tribulation HTTP/1.1" 200 7060 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
199.21.99.83 - - [30/Jun/2013:07:17:15 -0500] "GET /searchme/index.php?f=proud HTTP/1.1" 200 7080 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
199.21.99.83 - - [30/Jun/2013:07:17:34 -0500] "GET /searchme/index.php?f=soul HTTP/1.1" 200 7063 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
199.21.99.83 - - [30/Jun/2013:07:17:38 -0500] "GET /searchme/index.php?f=the_flesh_lusteth HTTP/1.1" 200 6951 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.c

2 个答案:

答案 0 :(得分:1)

你有太多的循环。您不需要输入logdate 循环。你已经遍历每一行了。

删除 for logdate ,并删除信息分配和打印报表。

(评论已经提到了这一点。)

答案 1 :(得分:1)

由于示例输出,问题有点令人困惑,但我很确定你想要这样的东西:

def unique_ips():
    f = open('logfile','r')
    ips = {}
    # This for loop collects all of the ips with their associated errors
    for line in f:
        ip = line.split()[0]
        try:
            ips[ip].append(line)
        except KeyError:
            ips[ip] = [line]

    # This for loop goes through all the ips that were collected
    # and prints out all errors for those ips
    for ip, errors in ips.iteritems():
        print ip
        errors.sort()
        for e in errors:
           logdate = e.split()[3]
           print "\t", logdate

           info = e.split()[5:11] 
           print "\t\t", info

    f.close()

从样本文件中生成此输出:

199.21.99.83
    [30/Jun/2013:07:16:13
        ['"GET', '/searchme/index.php?f=being_fruitful', 'HTTP/1.1"', '200', '7526', '"-"']
    [30/Jun/2013:07:16:45
        ['"GET', '/searchme/index.php?f=comparing_themselves', 'HTTP/1.1"', '200', '7369', '"-"']
    [30/Jun/2013:07:17:00
        ['"GET', '/searchme/index.php?f=tribulation', 'HTTP/1.1"', '200', '7060', '"-"']
    [30/Jun/2013:07:17:15
        ['"GET', '/searchme/index.php?f=proud', 'HTTP/1.1"', '200', '7080', '"-"']
    [30/Jun/2013:07:17:34
        ['"GET', '/searchme/index.php?f=soul', 'HTTP/1.1"', '200', '7063', '"-"']
    [30/Jun/2013:07:17:38
        ['"GET', '/searchme/index.php?f=the_flesh_lusteth', 'HTTP/1.1"', '200', '6951', '"-"']
66.249.73.40
    [30/Jun/2013:07:16:56
        ['"GET', '/espanol/displayAncient.cgi?ref=isa%2054:3', 'HTTP/1.1"', '500', '167', '"-"']
99.21.99.83
    [30/Jun/2013:07:15:50
        ['"GET', '/lenny/index.php?f=13', 'HTTP/1.1"', '200', '11244', '"-"']