从文件中提取网址

时间:2019-02-26 13:45:52

标签: python url filter

我正在尝试从具有以下格式的文件中提取URL。

[CertSpotter]     wwwqa.xyz.abc.com,1.1.1.1
[CertSpotter]     origin.xyz.abc.com,1.1.1.1
[CertSpotter]     wwwqa.xyz.abc.com,1.1.1.1
[CertSpotter]     wwwmg4.xyz.abc.com,1.1.1.1

我找到了python脚本,但是在那儿,我同时获得了URL和IP,但是我只需要一个URL。

import re

file_path = input("Enter the File Path: ")
f = open(file_path, 'r')
raw_text= str(f.readlines())
f.close()

domain = r"\b((?:https?://)?(?:(?:www\.)?(?:[\da-z\.-]+)\.(?:[a-z]{2,6})|(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)|(?:(?:[0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|(?:[0-9a-fA-F]{1,4}:){1,7}:|(?:[0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|(?:[0-9a-fA-F]{1,4}:){1,5}(?::[0-9a-fA-F]{1,4}){1,2}|(?:[0-9a-fA-F]{1,4}:){1,4}(?::[0-9a-fA-F]{1,4}){1,3}|(?:[0-9a-fA-F]{1,4}:){1,3}(?::[0-9a-fA-F]{1,4}){1,4}|(?:[0-9a-fA-F]{1,4}:){1,2}(?::[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:(?:(?::[0-9a-fA-F]{1,4}){1,6})|:(?:(?::[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(?::[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(?:ffff(?::0{1,4}){0,1}:){0,1}(?:(?:25[0-5]|(?:2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(?:25[0-5]|(?:2[0-4]|1{0,1}[0-9]){0,1}[0-9])|(?:[0-9a-fA-F]{1,4}:){1,4}:(?:(?:25[0-5]|(?:2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(?:25[0-5]|(?:2[0-4]|1{0,1}[0-9]){0,1}[0-9])))(?::[0-9]{1,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])?(?:/[\w\.-]*)*/?)\b"
foundip = re.findall( domain, raw_text )
for ip in foundip:
      print(ip)

运行脚本后,我得到以下输出。

wwwqa.xyz.abc.com
1.1.1.1
origin.xyz.abc.com
1.1.1.1
wwwmg4.xyz.abc.com
1.1.1.1

所需的输出。

wwwqa.xyz.abc.com
origin.xyz.abc.com
wwwmg4.xyz.abc.com

有人可以帮我解决这个问题吗?

谢谢

3 个答案:

答案 0 :(得分:0)

没有正则表达式。仅使用str方法。

例如:

with open(filename) as infile:
    for line in infile:
        val = line.strip().split()[-1].split(",")[0]
        print(val)

输出:

wwwqa.xyz.abc.com
origin.xyz.abc.com
wwwqa.xyz.abc.com
wwwmg4.xyz.abc.com

答案 1 :(得分:0)

import re
with open('file.txt') as f:
    result = re.findall(' +(.*),', f.read())

输出:

['wwwqa.xyz.abc.com', 'origin.xyz.abc.com', 'wwwqa.xyz.abc.com', 'wwwmg4.xyz.abc.com']

答案 2 :(得分:0)

import re

f = open('test.txt', 'r')
content = f.read()
pattern = r"^\[.*\]\s*(.*),.*"
matches = re.findall(pattern, content, re.MULTILINE|re.IGNORECASE)
print(matches)

输出:

['wwwqa.xyz.abc.com', 'origin.xyz.abc.com', 'wwwqa.xyz.abc.com', 'wwwmg4.xyz.abc.com']