Question

我想玩python来学习它，所以我正在开展一个小项目，但其中一部分需要我在这个列表中搜索一个名字：

https://bughunter.withgoogle.com/characterlist/1

（每次搜索名称时，第一个加1）

所以我将HTML抓取它，我是python的新手，如果有人能给我一个如何使这项工作的例子，我将不胜感激。

Answer 1

试试这个。您需要先安装bs4（python 3）。它将在网站页面上获得所有人的姓名：

from bs4 import BeautifulSoup as soup
import urllib.request
text=str(urllib.request.urlopen('https://bughunter.withgoogle.com/characterlist/1').read())
text=soup(text)
print(text.findAll(class_='item-list')[0].get_text())

Answer 2

import json
import requests
from bs4 import BeautifulSoup

URL = 'https://bughunter.withgoogle.com'


def get_page_html(page_num):
    r = requests.get('{}/characterlist/{}'.format(URL, page_num))
    r.raise_for_status()
    return r.text


def get_page_profiles(page_html):
    page_profiles = {}
    soup = BeautifulSoup(page_html)
    for table_cell in soup.find_all('td'):
        profile_name = table_cell.find_next('h2').text
        profile_url = table_cell.find_next('a')['href']
        page_profiles[profile_name] = '{}{}'.format(URL, profile_url)
    return page_profiles


if __name__ == '__main__':
    all_profiles = {}
    for page_number in range(1, 81):
        current_page_html = get_page_html(page_number)
        current_page_profiles = get_page_profiles(current_page_html)
        all_profiles.update(current_page_profiles)
    with open('google_hall_of_fame_profiles.json', 'w') as f:
        json.dump(all_profiles, f, indent=2)

Your question wasn't clear about how you wanted the data structured after scraping so I just saved the profiles in a dict (with the key/value pair as {profile_name: profile_url}) and then dumped the results to a json file.

Let me know if anything is unclear!

在HTML页面中搜索特定文本？

2 个答案: