使用Python 3和beautifulsoup4解析HTML表

时间:2018-07-10 15:51:58

标签: python python-3.x web-scraping beautifulsoup html-parsing

因此,我有用于解析数据的HTML表,并将其打印在控制台中。

HTML代码:

<table class='hdr'><tbody>
<tr><td class='hdr_name'><b>Station ID: </b></td><td class='hdr_value'><b>TUA1-W03281</b></td></tr>
<tr><td class='hdr_name'><b>Serial Number: </b></td><td class='hdr_value'><b>C737201011071281   Nest Number : RFID = N1111003 TS = N0000001</b></td></tr>
<tr><td class='hdr_name'><b>Date: </b></td><td class='hdr_value'><b>Wednesday, July 04, 2018</b></td></tr>
<tr><td class='hdr_name'><b>Time: </b></td><td class='hdr_value'><b>9:26:25 AM</b></td></tr>
<tr><td class='hdr_name'><b>Operator: </b></td><td class='hdr_value'><b>No operator logged in</b></td></tr>
<tr><td class='hdr_name'><b>Execution Time: </b></td><td class='hdr_value'><b>118.3051362 seconds</b></td></tr>
<tr><td class='hdr_name'><b>Number of Results: </b></td><td class='hdr_value'><b>169</b></td></tr>
<tr><td class='hdr_name'><b>UUT Result: </b></td><td class='hdr_value'><b><span style="color:#008000;">Passed</span></b></td></tr>
</tbody></table>

我的代码来抓取数据:

from datetime import datetime
from bs4 import BeautifulSoup

testURL = open('Report - C737201011071281 - 04-07-18-92626 AM.128144458.html')
soup = BeautifulSoup(testURL, "html.parser")

'Station ID'
station = soup.find("td", text="Station ID: ").find_next_sibling("td").text
print("\nStation ID:    " + station)

'Serial Number'
serialNumber = soup.find("td", text="Serial Number: ").find_next_sibling("td").text
cNumber = serialNumber[0:15]
print("Serial Number:    " + cNumber)

'Date'
date = soup.find("td", text="Date: ").find_next_sibling("td").text
datetime_object = datetime.strptime(date, '%A, %B %d, %Y')
print("Date:    " + datetime_object.strftime('%Y-%m-%d'))

'Time'
time = soup.find("td", text="Time: ").find_next_sibling("td").text
time_object = datetime.strptime(time, '%I:%M:%S AM')
print("Time:    " + time_object.strftime('%H:%M:%S'))

'Result'
result = soup.find("td", text="UUT Result: ").find_next_sibling("td").text
print("UUT Result:    " + result)

稍后在HTML中使用的表没有类或ID,我要做的是能够遍历每一行,直到找到其中包含单词“ Focus”的行。找到该行后,我要遍历该行下面的每一行,直到到达其中包含单词“ Result”的行。

下面是有关如何创建表格的HTML:

<tr><td class='label' colspan='2' style="border-color:#000000;background-color:#FFFFFF;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Focus:</td>
</tr>

<tr><td class='label' style="border-color:#000000;background-color:#FFFFFF;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Centre_Score:</td>
<td class='value' style="border-color:#000000;background-color:#32FFFF;"><span style='white-space:nowrap;'>0.08565902709961</span></td></tr>
<tr><td class='label' style="border-color:#000000;background-color:#FFFFFF;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Top_Left_Score:</td>
<td class='value' style="border-color:#000000;background-color:#32FFFF;"><span style='white-space:nowrap;'>0.06830544769764</span></td></tr>
<tr><td class='label' style="border-color:#000000;background-color:#FFFFFF;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Top_Right_Score:</td>
<td class='value' style="border-color:#000000;background-color:#32FFFF;"><span style='white-space:nowrap;'>0.06978666782379</span></td></tr>
<tr><td class='label' style="border-color:#000000;background-color:#FFFFFF;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Bottom_Right_Score:</td>
<td class='value' style="border-color:#000000;background-color:#32FFFF;"><span style='white-space:nowrap;'>0.06947886943817</span></td></tr>
<tr><td class='label' style="border-color:#000000;background-color:#FFFFFF;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Bottom_Left_Score:</td>
<td class='value' style="border-color:#000000;background-color:#32FFFF;"><span style='white-space:nowrap;'>0.07671363651752</span></td></tr>
<tr><td class='label' style="border-color:#000000;background-color:#FFFFFF;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Result:</td>
<td class='value' style="border-color:#000000;background-color:#32FFFF;"><span style='white-space:nowrap;'>1</span></td></tr>

如何遍历具有文本“ Focus:”的行之后的每一行,并将每个参数的值存储在名为“ Focus Results”的数组中。 我是Python的新手,希望能得到任何反馈,尤其是对我创建的.py脚本的反馈。

2 个答案:

答案 0 :(得分:0)

也许您想做这样的事情: 您首先搜索“焦点:”,然后遍历其父级(其他行)的兄弟姐妹。您可以在res字典中获得所需的结果。

all_labels = soup.find_all("td", class_="label")
res={}
for label in all_labels:
    if 'Focus' in label.text:
        for sib in label.parent.next_siblings:
            if isinstance(sib, Tag):
                lab = sib.find('td', class_='label').text.strip()
                val = sib.find('td', class_='value').text.strip()
                res[lab] = val
                if 'Result' in lab: break
        break;
print res

答案 1 :(得分:0)

您可以尝试这样获得预期的输出:

for items in soup.find_all("td"):
    if "Focus" in items.text:
        data = items.find_parent("tr")
        for elem in data.find_next_siblings():
            if "Result" in elem.find(class_="label").text:break
            item = elem.find(class_="label").get_text(strip=True)
            value = elem.find(class_="value").get_text(strip=True)
            print(f'{item} {value}')

输出:

Centre_Score: 0.08565902709961
Top_Left_Score: 0.06830544769764
Top_Right_Score: 0.06978666782379
Bottom_Right_Score: 0.06947886943817
Bottom_Left_Score: 0.07671363651752