Question

我有sever.log文件。我的正则表达式是提取所有由点分隔的3位数字。我的代码在下面，

192.168.10.20 - - [18/Jul/2017:08:41:37 +0000] "PUT /search/tag/list HTTP/1.0" 200 5042 "http://cooper.com/homepage/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/5342 (KHTML, like Gecko) Chrome/14.0.870.0 Safari/5342"
10.30.24.3 - - [18/Jul/2017:08:45:15 +0000] "POST /search/tag/list HTTP/1.0" 200 4939 "http://www.cole-brown.net/category/main/list/privacy/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/5322 (KHTML, like Gecko) Chrome/14.0.843.0 Safari/5322"
98.5.45.3 - - [18/Jul/2017:08:45:49 +0000] "GET /apps/cart.jsp?appID=8471 HTTP/1.0" 200 4958 "http://knight-chase.com/post.jsp" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_7_3; rv:1.9.6.20) Gecko/2013-11-03 17:44:01 Firefox/3.8"

我的代码

import re
with open (r'C:\Users\ubuntu\Desktop\Tests\apache.log', 'r') as fr1:
    line1 = fr1.read()
regex = r"(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
#print(re.findall(regex, line1, re.DOTALL))
listofip = (re.findall(regex, line1))
result ={}
for i in listofip:
    result[i] = listofip.count(i)
result

我的输出

{'192.168.10.20': 1,
 '14.0.870.0': 1,
 '10.30.24.3': 1,
 '14.0.843.0': 1,
 '98.5.45.3': 1,
 '1.9.6.20': 1}

所需的输出

{'192.168.10.20': 1,
 '10.30.24.3': 1,
 '98.5.45.3': 1}

Answer 1

如果每行都有IP，则可以简单地逐行阅读并将其拆分并获得第一项：

#line1=r'''192.168.10.20 - - [18/Jul/2017:08:41:37 +0000] "PUT /search/tag/list HTTP/1.0" 200 5042 "http://cooper.com/homepage/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/5342 (KHTML, like Gecko) Chrome/14.0.870.0 Safari/5342"
#10.30.24.3 - - [18/Jul/2017:08:45:15 +0000] "POST /search/tag/list HTTP/1.0" 200 4939 "http://www.cole-brown.net/category/main/list/privacy/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/5322 (KHTML, like Gecko) Chrome/14.0.843.0 Safari/5322"
#98.5.45.3 - - [18/Jul/2017:08:45:49 +0000] "GET /apps/cart.jsp?appID=8471 HTTP/1.0" 200 4958 "http://knight-chase.com/post.jsp" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_7_3; rv:1.9.6.20) Gecko/2013-11-03 17:44:01 Firefox/3.8"
#98.5.45.3 - - [18/Jul/2017:08:45:49 +0000] "GET /apps/cart.jsp?appID=8471 HTTP/1.0" 200 4958 "http://knight-chase.com/post.jsp" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_7_3; rv:1.9.6.20) Gecko/2013-11-03 17:44:01 Firefox/3.8"'''
result ={}
with open (r'C:\Users\ubuntu\Desktop\Tests\apache.log', 'r') as fr1:
    for line in fr1:
        ip = line.split()[0]
        if ip in result:
            result[ip] += 1
        else:
            result[ip] = 1
print(result)
# => {'192.168.10.20': 1, '10.30.24.3': 1, '98.5.45.3': 2}

请参见the Python demo。

要仅使用regex 在行的开头获取IP，可以使用

r'(?m)^\d{1,3}(?:\.\d{1,3}){3}'

请参见regex demo。

请注意，在行首匹配的IP正则表达式（请参见this reference）更好

r'^(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(?:\.(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}'

甚至是这个，考虑到每个IP后都有一个空格：

r'^(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(?:\.(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}(?!\S)'

详细信息

(?m)^-一行的开头

\d{1,3}-1到3位数字

(?:\.\d{1,3}){3}-出现.和1至3位数字。

请参见Python demo：

import re line1=r'''192.168.10.20 - - [18/Jul/2017:08:41:37 +0000] "PUT /search/tag/list HTTP/1.0" 200 5042 "http://cooper.com/homepage/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/5342 (KHTML, like Gecko) Chrome/14.0.870.0 Safari/5342" 10.30.24.3 - - [18/Jul/2017:08:45:15 +0000] "POST /search/tag/list HTTP/1.0" 200 4939 "http://www.cole-brown.net/category/main/list/privacy/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/5322 (KHTML, like Gecko) Chrome/14.0.843.0 Safari/5322" 98.5.45.3 - - [18/Jul/2017:08:45:49 +0000] "GET /apps/cart.jsp?appID=8471 HTTP/1.0" 200 4958 "http://knight-chase.com/post.jsp" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_7_3; rv:1.9.6.20) Gecko/2013-11-03 17:44:01 Firefox/3.8"''' rx = r"^\d{1,3}(?:\.\d{1,3}){3}\b" listofip = re.findall(rx, line1, re.M) result ={} for ip in listofip: if ip in result: result[ip] += 1 else: result[ip] = 1 print(result) # => {'192.168.10.20': 1, '10.30.24.3': 1, '98.5.45.3': 1}

Answer 2

您的日志文件是CSV文件，并且IP地址在第一列中。为此，使用正则表达式是没有意义的。

import csv

with open('apache.log', encoding='utf8') as logfile:
    reader = csv.reader(logfile, delimiter=' ')

    for row in reader:
        print(row[0])

输出

192.168.10.20
10.30.24.3
98.5.45.3

Answer 3

您可以使用^和MULTILINE标志选项。
如果要计算列表中的元素，也可以使用Counter。

测试代码

import re
from collections import Counter

line1 = '''
192.168.10.20 - - [18/Jul/2017:08:41:37 +0000] "PUT /search/tag/list HTTP/1.0" 200 5042 "http://cooper.com/homepage/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/5342 (KHTML, like Gecko) Chrome/14.0.870.0 Safari/5342"
10.30.24.3 - - [18/Jul/2017:08:45:15 +0000] "POST /search/tag/list HTTP/1.0" 200 4939 "http://www.cole-brown.net/category/main/list/privacy/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/5322 (KHTML, like Gecko) Chrome/14.0.843.0 Safari/5322"
98.5.45.3 - - [18/Jul/2017:08:45:49 +0000] "GET /apps/cart.jsp?appID=8471 HTTP/1.0" 200 4958 "http://knight-chase.com/post.jsp" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_7_3; rv:1.9.6.20) Gecko/2013-11-03 17:44:01 Firefox/3.8"
'''

regex = r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}'
matches = re.findall(regex, line1, flags=re.MULTILINE)

print(dict(Counter(matches)))

输出

{'192.168.10.20': 1, '10.30.24.3': 1, '98.5.45.3': 1}

正则表达式从日志文件中提取不需要的IP地址

3 个答案: