读取跨越行的数据的HTML表

Question

我使用BeautifulSoup提取了一个HTML表格，并希望将其导入到pandas DataFrame中。但是，原始表中的数据分布在多行中。以下是两个参考条目：

＆＃13;

＆＃13;

<table>
    
    <tbody><tr>
      <td>Record : 1 of 749</td>
      </tr>
    <tr>
      <td width="111">Patients Name</td>
        <td width="4">:</td>
        <td colspan="4">Andrew Smith</td>
        </tr>
    <tr>
      <td>Admit Date</td>
      <td>:</td>
      <td width="189">20-MAR-2018</td>
      <td>Group Number </td>
      <td>:</td>
      <td>17</td>
    </tr>
    <tr>
      <td>Address</td>
        <td>:</td>
        <td>123 Sunshine Ave </td>
        <td>Postal Code </td>
        <td>:</td>
        <td>12345</td>
    </tr>
    <tr>
      <td>Blood Type</td>
        <td>:</td>
        <td>A	</td>
        <td width="96">Ward Class</td>
        <td width="4">:</td>
        <td width="174">A</td>
      </tr>
    <tr>
      <td>Age</td>
        <td>:</td>
        <td>45</td>
        <td>Height</td>
        <td>:</td>
        <td>
		174cm	
		</td>
      </tr>
    <tr>
      <td>Weight</td>
        <td>:</td>
        <td>102kg</td>
        <td>ID</td>
        <td>:</td>
        <td>
		013</td>
      </tr>
    <tr>
      <td><hr/></td>
        </tr>
    
    <tr>
      <td>Record : 2 of 749</td>
      </tr>
    <tr>
      <td width="111">Patients Name</td>
        <td width="4">:</td>
        <td colspan="4">Margaret Chow</td>
        </tr>
    <tr>
      <td>Admit Date</td>
      <td>:</td>
      <td width="189">19-MAR-2018</td>
      <td>Group Number </td>
      <td>:</td>
      <td>14</td>
    </tr>
    <tr>
      <td>Address</td>
        <td>:</td>
        <td>5 Mango Beach </td>
        <td>Postal Code </td>
        <td>:</td>
        <td>54321</td>
    </tr>
    <tr>
      <td>Blood Type</td>
        <td>:</td>
        <td>B	</td>
        <td width="96">Ward Class</td>
        <td width="4">:</td>
        <td width="174">B2</td>
      </tr>
    <tr>
      <td>Age</td>
        <td>:</td>
        <td>32</td>
        <td>Height</td>
        <td>:</td>
        <td>
		154cm	
		</td>
      </tr>
    <tr>
      <td>Weight</td>
        <td>:</td>
        <td>52kg</td>
        <td>ID</td>
        <td>:</td>
        <td>
		051</td>
      </tr>
    <tr>
      <td><hr/></td>
        </tr>
    
  </tbody></table>

＆＃13;

＆＃13;

＆＃13;

我使用以下代码将上表提取到pandas DataFrame中：

import pandas as pd
table = str(table)
df = pd.read_html(table)
df = pd.DataFrame(df)
df

我的df看起来像这样：

但我希望它是一个DataFrame，其中包含[[＆＃39;患者姓名＆＃39;，＆＃39;承认日期＆＃39;，＆＃39;群组编号＆＃39;，＆ ＃39;地址＆＃39;，＆＃39;邮政编码＆＃39;血型＆＃39;，＆＃39;病房类＆＃39;，＆＃39;年龄＆＃39;，＆＃39; ;身高＆＃39;，＆＃39;体重＆＃39;，＆＃39; ID＆＃39;]。

对此我不熟悉。非常感谢任何建议！

Answer 1

import pandas as pd
from bs4 import BeautifulSoup as bs
soup = bs(table, 'html.parser')
df = pd.DataFrame() # you can add index and column details at this point too
row_index = -1
for row in soup.find_all('tr'):
    if row.find('td').find('hr'): # few rows has a horizontal line; skipping them
        continue
    if len(row.find_all('td')) == 1: # skipping the row stating Record : 1 of ...
    #if row.find_all('td')[0].get_text().startswith('Record :'):
        row_index += 1
        continue
    tds = [td.get_text().strip() for td in row.find_all('td')]
    df.at[row_index, tds[0]] = tds[2]
    if len(tds) > 3: #few rows have multiple tds; might have to make this dynamic if its more than 2 fields per row
        df.at[row_index, tds[3]] = tds[5]

这是我第一次使用网络抓取工具，我很高兴找到解决方案！这段代码适用于您定义的问题。您可能必须根据表结构更改某些条件。

PS：这是我在Stack Overflow的第一个答案，我真的希望这会有所帮助：）