如果代码具有属性

时间：2019-01-07 15:43:51

标签： python pandas beautifulsoup

我想从表中抓取数据，如果行中有<tr>，则取整行<td BGCOLOR="#D42A2A">

html看起来像这样（多于2行）：

<tr bgcolor="#f4f4f4">
<td height="25" nowrap="NOWRAP">&nbsp;ITEM_1&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;07:58:46&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;Connected&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;07:58:00&nbsp;</td>
<td height="25" nowrap="NOWRAP" bgcolor="#55aa2a">&nbsp;--:--:--&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;01:25:00 &nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp; 22:00:00&nbsp;</td>
</tr>
<tr bgcolor="#ffffff">
<td height="25" nowrap="NOWRAP">&nbsp;ITEM_2&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;07:58:46&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;Connected&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;191&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;07:58:01&nbsp;</td>
<td height="25" nowrap="NOWRAP" bgcolor="#55aa2a">&nbsp;--:--:--&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;01:25:00 &nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp; 22:00:00&nbsp;</td>
</tr>
<tr bgcolor="#ffffff">
<td height="25" nowrap="NOWRAP">&nbsp;ITEM_3&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;07:59:02&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;Connected&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;36&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;36&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;07:58:01&nbsp;</td>
<td height="25" nowrap="NOWRAP" bgcolor="#d42a2a">&nbsp;--:--:--&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;03:10:00 &nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp; 22:00:00&nbsp;</td>
</tr>

我用过this，但这里的答案给出了表中的所有行，而不是包含必要属性的行

到目前为止，我的代码如下：

data = []

rows = table_body.find_all('tr')
for row in rows:
    cols = row.find_all('td')
    cols = [ele.text.strip() for ele in cols]
    data.append([ele for ele in cols if ele])

然后我再次抓取该网站以找到bgcolor属性，将其添加到列表中，将列表追加到框架中，并删除没有正确bgcolor的任何行。

这都是效率很低的

仅当行的td.attrs中存在bgcolor时，如何才能刮取html以从表中获取行

编辑：将以下解决方案应用于整个html之后，脚本将返回空列表（这是我的错，因为不包括更多html）。下面的html是更完整的版本，其中包含更多标签。 

<html><head><title></title><style type="text/css">
BODY {
font-family: Tahoma, Verdana, Geneva, Arial, Helvetica, sans-serif;
font-size: 11px;
background-color: #FFFFFF
;}TABLE {
font-family: Tahoma, Verdana, Geneva, Arial, Helvetica, sans-serif;
font-size: 11px;
background-color: #FFFFFF;}
DIV.boldText {
font-size: 11px;font-weight: bold;
}
</style>
<meta http-equiv="REFRESH" content="10">
</head><body>
<form name="DataViewChooser">
<hr width="95%" align="CENTER" color="#55aa2a">
<table width="95%" align="CENTER">
<tbody><tr><td width="40" height="65" title="(c) ITEMS"><img 
src="/icons/geneos_logo.png"></td>
<td width="25" align="LEFT">
<img title="Refresh" style="cursor: hand;" onclick="reloadPage()" 
src="/icons/refresh.png"></td>
<td width="25" title="Show Fail and Warning Only" align="LEFT"><img 
style="cursor: hand;" onclick="userContractView()" src="/icons/minimise.png"></td>
<td width="25" align="LEFT"><img title="Home" style="cursor: hand;" onclick="goHome()" src="/icons/up.png"></td>
<td align="RIGHT" nowrap="NOWRAP"><img src="/icons/hostgreen.gif">
<div class="boldText">&nbsp;DASHBOARD-CV_AMER_Dashboard</div>&nbsp; [GROUP]
</td>
</tr></tbody></table><hr width="95%" align="CENTER" color="#55aa2a"></form>
<br><table width="95%" align="CENTER"><tbody><tr><td><table>
<tbody><tr><th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;AMER&nbsp; 
</th>
<td nowrap="NOWRAP" bgcolor="#55aa2a">&nbsp;&nbsp;</td></tr>
</tbody></table></td></tr></tbody></table>
<br><table width="99%" align="CENTER">
<tbody><tr bgcolor="#c0c0c0">
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;RowName&nbsp;</th>
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;Gateway_updatetime&nbsp; 
</th>
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;Gateway_state&nbsp;</th>
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;OrdersCleared&nbsp;</th>
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;Ticketsread&nbsp;</th>
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;OrdersNotCleared&nbsp; 
</th>
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;TicketsNotCleared&nbsp; 
</th>
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;LastReadingtime&nbsp; 
</th>
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;LastClearingtime&nbsp; 
</th>
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;ClearingInProgress&nbsp; 
</th>
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;YestVolumes&nbsp;</th>
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;Starttime&nbsp;</th>
<th height="20" align="LEFT" nowrap="NOWRAP">&nbsp;Stoptime&nbsp;</th>
</tr><tr bgcolor="#f4f4f4">
<td height="25" nowrap="NOWRAP">&nbsp;ITEM_4&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;07:58:46&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;Connected&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;07:58:00&nbsp;</td>
<td height="25" nowrap="NOWRAP" bgcolor="#d42a2a">&nbsp;--:--:--&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;01:25:00 &nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp; 22:00:00&nbsp;</td>
</tr>
<tr bgcolor="#ffffff">
<td height="25" nowrap="NOWRAP">&nbsp;ITEM_5&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;07:58:46&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;Connected&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;191&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;07:58:01&nbsp;</td>
<td height="25" nowrap="NOWRAP" bgcolor="#55aa2a">&nbsp;--:--:--&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;0&nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp;01:25:00 &nbsp;</td>
<td height="25" nowrap="NOWRAP">&nbsp; 22:00:00&nbsp;</td>
</tr>
</tbody></table><script language="JavaScript" src="/cookie.js"></script>
</body></html>'''

值得注意的是，我正在使用urllib.request打开url，然后与BS进行解析

3 个答案:

答案 0 :(得分：1)

您可以使用any：

from bs4 import BeautifulSoup as soup
d = soup(content, 'html.parser')
results = [i for i in d.find_all('tr') if any(c.attrs.get('bgcolor') == "#d42a2a" for c in i.find_all('td'))]

输出：

[<tr bgcolor="#ffffff">
  <td height="25" nowrap="NOWRAP"> ITEM_3 </td>
  <td height="25" nowrap="NOWRAP"> 07:59:02 </td>
  <td height="25" nowrap="NOWRAP"> Connected </td>
  <td height="25" nowrap="NOWRAP"> 0 </td>
  <td height="25" nowrap="NOWRAP"> 36 </td>
  <td height="25" nowrap="NOWRAP"> 0 </td>
  <td height="25" nowrap="NOWRAP"> 36 </td>
  <td height="25" nowrap="NOWRAP"> 07:58:01 </td>
  <td bgcolor="#d42a2a" height="25" nowrap="NOWRAP"> --:--:-- </td>
  <td height="25" nowrap="NOWRAP"> 0 </td>
  <td height="25" nowrap="NOWRAP"> 0 </td>
  <td height="25" nowrap="NOWRAP"> 03:10:00  </td>
  <td height="25" nowrap="NOWRAP">  22:00:00 </td>
 </tr>]

答案 1 :(得分：1)

您可以应用searching function，在其中可以检查标记的名称为tr，以及检查该行是否包含带有td的{​​{1}}元素： 

bgcolor="#D42A2A"

您当然可以直接在列表理解中执行相同的检查：

def rows_with_desired_bgcolor(elm):
    return elm.name == 'tr' and elm.find('td', bgcolor="#D42A2A")

table_body.find_all(rows_with_desired_bgcolor)

其中[tr for tr in table_body('tr') if tr.find('td', bgcolor="#D42A2A")] 是table_body('tr')的快捷方式。

答案 2 :(得分：0)

找到所有td包含bgcolor="#d42a2a"的内容，然后选择.parent 

cells = table_body.find_all('td', bgcolor="#d42a2a")
for cell in cells:
    print(cell.parent) 
    # <tr>...<td bgcolor="#d42a2a">...</tr>
相关问题
最新问题