我正在尝试抓取本网站中的地址- https://aca.accela.com/ALAMEDA/Cap/CapHome.aspx?module=Building&TabName=Building
这是HTML的一部分的外观-
<tr class="ACA_TabRow_Odd ACA_TabRow_Odd_FontSize">
<td class="ACA_AlignLeftOrRightTop"><input id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_CB_0" type="checkbox" name="ctl00$PlaceHolderMain$dgvPermitList$gdvPermitList$ctl02$CB_0" onclick="Check(this,'ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_hfSaveSelectedItems');" class="aca_gridview_checkbox" title="To select one record in current page."></td><td class="ACA_AlignLeftOrRightTop" style="width:60px;">
<div style="margin-right: 5px;">
<span id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_lblUpdatedTime" css="ACA_SmLabel ACA_SmLabel_FontSize">07/10/2018</span>
</div>
</td><td class="ACA_AlignLeftOrRightTop" style="width:110px;">
<div>
<strong>
</strong>
<a id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_hlPermitNumber" href="/ALAMEDA/Cap/CapDetail.aspx?Module=Building&TabName=Building&capID1=18SPV&capID2=00000&capID3=00039&agencyCode=ALAMEDA&IsToShowInspection=">
<strong>
<span id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_lblPermitNumber1">SPV18-0037</span></strong>
</a>
<input type="hidden" id="RecordId" value="18SPV-00000-00039">
</div>
</td><td class="ACA_AlignLeftOrRightTop" style="width:130px;">
<div class="ACA_CapListStyle">
<span id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_lblType">Solar Photovoltaic System Residential</span>
</div>
</td><td class="ACA_AlignLeftOrRightTop" style="width:130px;">
<div class="ACA_CapListStyle">
<span id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_lblDescription">INSTALL A 5.12 KW SOLAR SYSTEM, ROOFTOP, FLUSH MOUNT, 16 PANELS (BLDG)</span>
</div>
</td><td class="ACA_AlignLeftOrRightTop" style="width:130px;">
<div class="ACA_CapListStyle">
<span id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_lblAddress">1623 CLINTON AVE, ALAMEDA CA 94501</span>
</div>
</td><td class="ACA_AlignLeftOrRightTop" style="width:100px;">
<div style="white-space: nowrap;" class="ACA_CapListStyle">
<div id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_panelStatus">
<span id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_lblStatus">Plan Review</span>
</div>
<div id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_panelbtnRenewalDetail">
</div>
</div>
</td><td class="ACA_AlignLeftOrRightTop" style="width:100px;">
<div style="white-space: nowrap;" class="ACA_CapListStyle">
<div id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_Panel2">
</div>
<div id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_Panel3">
</div>
<div id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_Panel4">
</div>
<div id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_Panel5">
</div>
</div>
</td><td class="ACA_AlignLeftOrRightTop" style="width:110px;">
<div class="ACA_CapListStyle">
<span id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_lblShortNote">INSTALL A 5.12 KW SOLAR SYSTEM, ROOFTOP, FLUSH MOU</span>
</div>
</td><td class="ACA_Hide">
<div class="ACA_CapListStyle">
<span id="ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl02_lblPermitAddress">1623 CLINTON AVE, ALAMEDA CA 94501</span>
</div>
</td>
</tr>
我正在尝试从td列获取地址。 这是我在python中的代码-
import bs4
from bs4 import BeautifulSoup
try:
from urllib.request import urlopen
except ImportError:
from urllib2 import urlopen
import os
from lxml import html
import requests
import csv
import sys
for page_no in range(2, 50):
curr_page = str(page_no)
if page_no < 9 :
data = {
'ctl00$ScriptManager1': 'ctl00$PlaceHolderMain$dgvPermitList$updatePanel|ctl00$PlaceHolderMain$dgvPermitList$gdvPermitList$ctl13$ctl0'+curr_page,
'ctl00$PlaceHolderMain$generalSearchForm$ddlGSPermitType': 'Building/Solar Photovoltaic System/Residential/NA',
'ctl00$PlaceHolderMain$generalSearchForm$txtGSStartDate': '01/01/2008',
'ctl00$PlaceHolderMain$generalSearchForm$txtGSEndDate': '07/17/2018'
}
else :
data = {
'ctl00$ScriptManager1': 'ctl00$PlaceHolderMain$dgvPermitList$updatePanel|ctl00$PlaceHolderMain$dgvPermitList$gdvPermitList$ctl13$ctl1'+curr_page,
'ctl00$PlaceHolderMain$generalSearchForm$ddlGSPermitType': 'Building/Solar Photovoltaic System/Residential/NA',
'ctl00$PlaceHolderMain$generalSearchForm$txtGSStartDate': '01/01/2008',
'ctl00$PlaceHolderMain$generalSearchForm$txtGSEndDate': '07/17/2018'
}
page = requests.get('https://aca.accela.com/ALAMEDA/Cap/CapHome.aspx?module=Building&TabName=Building', data = data)
soup = BeautifulSoup(page.text, 'html.parser')
print('Page ', page_no)
#print(soup)
address = soup.find_all('tbody')
print(address)
当我尝试打印地址变量时,它仅返回一个空字符串。当我尝试打印汤变量时,它返回网站的整个代码,而不是我们单击搜索时找到的数据。有人可以指导我哪里出问题了吗?
答案 0 :(得分:0)
find_all的方法签名为find_all(name,attrs,recursive,string,limit,**kwargs)
您以span
的形式传递名称,但随后尝试使用字符串作为关键字参数id
,这是不正确的。您可能想要的是
id = 'ctl00_PlaceHolderMain_dgvPermitList_gdvPermitList_ctl09_lblAddress'
作为第二个论点。