返回具体内容

时间:2015-09-23 17:07:15

标签: python web-scraping beautifulsoup

我只需要ip地址。如何废弃。我的代码现在 -

import urllib
import urllib.request
from bs4 import BeautifulSoup

x = urllib.request.urlopen('http://bannedhackersips.blogspot.com/2014_08_04_archive.html')
soup = BeautifulSoup(x,"html.parser")
data = soup.find_all("ul", {"class": "posts"})

for content in data:
   print(content.text)

输出:

[Fail2Ban] SSH: banned 116.10.191.162
[Fail2Ban] SSH: banned 116.10.191.204
[Fail2Ban] SSH: banned 61.174.51.232
[Fail2Ban] SSH: banned 61.174.51.224
[Fail2Ban] SSH: banned 116.10.191.225
[Fail2Ban] SSH: banned 200.162.47.130
[Fail2Ban] SSH: banned 116.10.191.175
[Fail2Ban] SSH: banned 61.174.51.223
[Fail2Ban] SSH: banned 61.174.51.234
[Fail2Ban] SSH: banned 61.174.51.209
[Fail2Ban] SSH: banned 116.10.191.165
[Fail2Ban] SSH: banned 106.240.247.220

1 个答案:

答案 0 :(得分:2)

您可以使用正则表达式从文本中提取:

data = soup.find("ul", {"class": "posts"})

import re

r = re.compile("\d+\.\d+\.\d+\.\d+")

print(r.findall(data.text))
['116.10.191.162', '116.10.191.204', '61.174.51.232', '61.174.51.224', '116.10.191.225', '200.162.47.130', '116.10.191.175', '61.174.51.223', '61.174.51.234', '61.174.51.209', '116.10.191.165', '106.240.247.220']

或者当模式重复时,您可以拆分为带有分割线的子串,并从每个子串的末尾拆分一次以提取ip:

data = soup.find("ul", {"class": "posts"})

ips = [line.rsplit(None, 1)[1] for line in data.text.splitlines() if line]

print(ips)
['116.10.191.162', '116.10.191.204', '61.174.51.232', '61.174.51.224', '116.10.191.225', '200.162.47.130', '116.10.191.175', '61.174.51.223', '61.174.51.234', '61.174.51.209', '116.10.191.165', '106.240.247.220']

页面上只有一个posts类,因此查找就足够了,当您迭代find_all时,实际上是在单个元素列表上进行迭代。