Question

我正在尝试从Basketball-reference.com（https://www.basketball-reference.com/leagues/NBA_1980.html）解析其他统计信息表。但是，我想解析的表在html注释内。

使用以下代码

html = requests.get("http://www.basketball-reference.com/leagues/NBA_2016.html").content
cleaned_soup = BeautifulSoup(re.sub("<!--|-->","", html))

结果如下

TypeError                                 Traceback (most recent call last)
<ipython-input-35-93508687bbc6> in <module>()
----> 1 cleaned_soup = BeautifulSoup(re.sub("<!--|-->","", html))

~/.pyenv/versions/3.7.0/lib/python3.7/re.py in sub(pattern, repl, string, count, flags)
    190     a callable, it's passed the Match object and must return
    191     a replacement string to be used."""
--> 192     return _compile(pattern, flags).sub(repl, string, count)
    193 
    194 def subn(pattern, repl, string, count=0, flags=0):

TypeError: cannot use a string pattern on a bytes-like object

我正在使用python3.7。

Answer 1

您可以尝试使用BeautifulSoup从HTML返回注释，而不是尝试使用[{_id:ObjectId(...),message: my AIRTIME is still valid}, {id:ObjectId(...),message: I doubt if AIRTIME could be used anymore}]将所有HTML放入注释中。然后，也可以使用BeautifulSoup解析这些内容，以根据需要提取任何表元素，例如：

re

这将使您在表中的行开始：

import requests
from bs4 import BeautifulSoup, Comment


html = requests.get("http://www.basketball-reference.com/leagues/NBA_2016.html").content
soup = BeautifulSoup(html, "html.parser")

for comment in soup.find_all(text=lambda t : isinstance(t, Comment)):
    comment_html = BeautifulSoup(comment, "html.parser")

    for table in comment_html.find_all("table"):
        for tr in table.find_all("tr"):
            row = [td.text for td in tr.find_all("td")]
            print(row)
        print()

注意：为避免获取['Finals', 'Cleveland Cavaliers \nover \nGolden State Warriors\n\xa0(4-3)\n', 'Series Stats'] ['\n\n\nGame 1\nThu, June 2\nCleveland Cavaliers\n89@ Golden State Warriors\n104\n\nGame 2\nSun, June 5\nCleveland Cavaliers\n77@ Golden State Warriors\n110\n\nGame 3\nWed, June 8\nGolden State Warriors\n90@ Cleveland Cavaliers\n120\n\nGame 4\nFri, June 10\nGolden State Warriors\n108@ Cleveland Cavaliers\n97\n\nGame 5\nMon, June 13\nCleveland Cavaliers\n112@ Golden State Warriors\n97\n\nGame 6\nThu, June 16\nGolden State Warriors\n101@ Cleveland Cavaliers\n115\n\nGame 7\nSun, June 19\nCleveland Cavaliers\n93@ Golden State Warriors\n89\n\n\n', 'Game 1', 'Thu, June 2', 'Cleveland Cavaliers', '89', '@ Golden State Warriors', '104', 'Game 2', 'Sun, June 5', 'Cleveland Cavaliers', '77', '@ Golden State Warriors', '110', 'Game 3', 'Wed, June 8', 'Golden State Warriors', '90', '@ Cleveland Cavaliers', '120', 'Game 4', 'Fri, June 10', 'Golden State Warriors', '108', '@ Cleveland Cavaliers', '97', 'Game 5', 'Mon, June 13', 'Cleveland Cavaliers', '112', '@ Golden State Warriors', '97', 'Game 6', 'Thu, June 16', 'Golden State Warriors', '101', '@ Cleveland Cavaliers', '115', 'Game 7', 'Sun, June 19', 'Cleveland Cavaliers', '93', '@ Golden State Warriors', '89'] ['Game 1', 'Thu, June 2', 'Cleveland Cavaliers', '89', '@ Golden State Warriors', '104'] ['Game 2', 'Sun, June 5', 'Cleveland Cavaliers', '77', '@ Golden State Warriors', '110'] ['Game 3', 'Wed, June 8', 'Golden State Warriors', '90', '@ Cleveland Cavaliers', '120'] ['Game 4', 'Fri, June 10', 'Golden State Warriors', '108', '@ Cleveland Cavaliers', '97']，您可以使用cannot use a string pattern on a bytes-like object而非.text将字符串传递给正则表达式。

从Basketball-reference.com解析NBA赛季统计数据，如何删除html注释标签

1 个答案: