我想点击一个按钮,在网站上加载更多数据。 我把一个time.sleep代码部分放在前后等待一段时间,但它不起作用。
此按钮html代码如下所示:
<button data-mode="arrivals" data-page="-1" data-timestamp="1506584673.386" ng-click="loadMoreFlights($event)" data-current-page="1" data-loading-text="<i class="fa fa-circle-o-notch fa-spin"></i> Loading earlier flights..." class="btn btn-table-action btn-flights-load">Load earlier flights</button>
我试过这个:
def scrape(urls):
browser = webdriver.Firefox()
datatable=[]
for url in urls:
browser.get(url)
time.sleep(5)
driver.find_element_by_xpath("//a[contains(.,'Load earlier flights')]")
time.sleep(5)
html = browser.page_source
soup=BeautifulSoup(html,"html.parser")
table = soup.find('table', { "class" : "table table-condensed table-hover data-table m-n-t-15" })
soup2=BeautifulSoup(html,"html.parser")
name = soup2.h2.string
soup3=BeautifulSoup(html,"html.parser")
name2 = soup3.h1.string
soup4=BeautifulSoup(html,"html.parser")
name3 = soup4.h3.string
name4 = datetime.now()
for record in table.find_all('tr', class_="hidden-xs hidden-sm ng-scope"):
temp_data = []
temp_data.append(name4)
temp_data.append(name)
temp_data.append(name2)
temp_data.append(name3)
for data in record.find_all("td"):
temp_data.append(data.text.encode('latin-1'))
newlist = filter(None, temp_data)
datatable.append(newlist)
time.sleep(10)
browser.close()
return datatable
为什么不起作用?
编辑:
Selenium版本是:3.5.0
Firefox版本为:52.3.0
答案 0 :(得分:0)
我认为问题是你的按钮的路径。它应该是:
driver.find_element_by_xpath('//button[contains(text(), "Load earlier flights")]').click()
答案 1 :(得分:0)
尝试使用此代码
driver.find_element_by_xpath('//button[contains(@class,'btn btn-table-action btn-flights-load')][contains(text(),'Load earlier flights')]').click();