beautifulsoup刮一个href标题

时间:2018-06-06 16:39:21

标签: python beautifulsoup

我希望用BeautifulSoup刮掉a-href中的标题。 实际上,我的代码没有正常工作。

import requests
from bs4 import BeautifulSoup
name = 'Flow'
namec = 'Flow'
url = 'http://warframe.wikia.com/wiki/' + name
r = requests.get(url)
soup = BeautifulSoup(r.text, 'html.parser')
main_text = soup.find('div', class_='mw-content-ltr mw-content-text')
ada = main_text.find_all('a', title=True)

页面为http://warframe.wikia.com/wiki/Flow

我需要从div类pi-data-value pi-font中提取数据,如下所示:

Missions:
Survival (DS3, T3)
Excavation (T3)
Enemies:
Arid Butcher 0.03%
Bailiff 0.03%
Bailiff Defector 0.03%
Butcher 0.03%
Drahk Master 0.03%
Drekar Manic 0.03%
Frontier Bailiff 0.03%
Frontier Butcher 0.03%
Grineer Manic 0.22%
Hyekka Master 0.03%
Infested Chroma 0.6%
Infested Mesa 0.6%
Kuva Butcher 0.03%
Kuva Drahk Master 0.03%
Kuva Hyekka Master 0.03%
Tenno Specter 0.6%
Tusk Butcher 0.03%
Other:
Orokin Tower Containers

2 个答案:

答案 0 :(得分:0)

这不是一个完整的解决方案,但它可以让你开始

import requests
from bs4 import BeautifulSoup

r = requests.get("http://warframe.wikia.com/wiki/Flow")
content = BeautifulSoup(r.text,'lxml')
main_div = content.find_all('div', {'class': ["pi-data-value", "pi-font"]})[3]
child_tags = main_div.findChildren()
for child in child_tags:
      print(child.text)

根据您的需要格式化文本

答案 1 :(得分:0)

你走了:

import requests
from bs4 import BeautifulSoup
name = 'Flow'
namec = 'Flow'
url = 'http://warframe.wikia.com/wiki/' + name
r = requests.get(url)
soup = BeautifulSoup(r.text, 'html.parser')
main_text = soup.find('div', id='mw-content-text').find_all('div', class_='pi-data-value pi-font')
for d in main_text:
    if d.find_next(string=True) == 'Missions:':
        snippets = str(d).split("<br/>")
        for snippet in snippets:
            snippet_soup = BeautifulSoup(snippet, 'html.parser')
            print(snippet_soup.text)