Question

from bs4 import BeautifulSoup
import re
import pandas as pd
import os
soup_level1=BeautifulSoup(driver.page_source, 'lxml')

在导入一些HTML（例如http://www.espncricinfo.com/series/18886/scorecard/1157372/）后，我意识到应该是一个表'，实际上不是表，因此似乎我需要自己构建表。

Batsmen = soup_level1.find_all('div',class_="cell batsmen")
pd.Series(Batsmen)

0     <div class="cell batsmen" data-reactid="182">B...
1     <div class="cell batsmen" data-reactid="191"><...
...
18    <div class="cell batsmen" data-reactid="541"><...
dtype: object

我可以使用以下方法提取蝙蝠侠的名字：

FirstBat = Batsmen[1]
FirstBat = str(FirstBat)
FirstBat = pd.Series(FirstBat)
FirstBat = FirstBat.str.split(pat = ">",expand=True)
FirstBat = FirstBat[2]
FirstBat

0    S Dhawan</a
Name: 2, dtype: object

然后我希望能够将击球手的名字加入他的统计数据，但是Stats [0：4]指的是标题，而Stats [5:10]指的是第一个击球手：

Stats = soup_level1.find_all('div',class_="cell runs")
pd.Series(Stats)
0     <div class="cell runs" data-reactid="184">R</div>
1     <div class="cell runs" data-reactid="185">B</div>
2     <div class="cell runs" data-reactid="186">4s</...
3     <div class="cell runs" data-reactid="187">6s</...
4     <div class="cell runs" data-reactid="188">SR</...
5     <div class="cell runs" data-reactid="194">4</div>
6     <div class="cell runs" data-reactid="195">8</div>
7     <div class="cell runs" data-reactid="196">1</div>
8     <div class="cell runs" data-reactid="197">0</div>
9     <div class="cell runs" data-reactid="198">50.0... 
...
94    <div class="cell runs" data-reactid="548">-</div>
Length: 95, dtype: object

能够添加看起来像这样的东西的最佳方法是什么？

    Batsmen  R  B  4s  6s    SR
0  S Dhawan  4  8   0   0  50.0

Answer 1

让您入门的事情：

from bs4 import BeautifulSoup
import numpy as np
import requests

html_doc= requests.get(r'http://www.espncricinfo.com/series/18886/scorecard/1157372/').content
soup = BeautifulSoup(html_doc, 'html.parser')

data = []
for div in soup.find_all('div',class_="cell runs"):
    data.append(div.text)

np.array(data).reshape(-1,5)

哪个输出

array([['R', 'B', '4s', '6s', 'SR'],
       ['14', '12', '2', '0', '116.66'],
       ['68', '55', '5', '1', '123.63'],
       ['39', '30', '2', '2', '130.00'],
       ['2', '3', '0', '0', '66.66'],
       ['9', '6', '1', '0', '150.00'],
       ['0', '1', '0', '0', '0.00'],
       ['0', '2', '0', '0', '0.00'],
       ['1', '2', '0', '0', '50.00'],
       ['0', '1', '0', '0', '0.00'],
       ['17', '8', '2', '1', '212.50'],
       ['R', 'B', '4s', '6s', 'SR'],
       ['0', '3', '0', '0', '0.00'],
       ['4', '2', '1', '0', '200.00'],
       ['14', '15', '2', '0', '93.33'],
       ['2', '7', '0', '0', '28.57'],
       ['0', '2', '0', '0', '0.00'],
       ['1', '3', '0', '0', '33.33'],
       ['19', '23', '2', '0', '82.60'],
       ['34', '29', '6', '0', '117.24'],
       ['3', '7', '0', '0', '42.85'],
       ['6', '3', '1', '0', '200.00'],
       ['2', '7', '0', '0', '28.57']], dtype='<U6')

从那里开始，将其放入数据帧实际上并不难。您将要小心，尽管它正在读取多个表，但是您可能会在途中看到第二个“标题”行（即['R', 'B', '4s', '6s', 'SR']），因此您需要决定要怎么做。

如何将HTML切片为数据框

1 个答案: