无法使用selenium PhantomJS点击按钮并获取数据

时间:2017-10-24 13:35:34

标签: python selenium phantomjs

我试图从预订中获取一些数据。但是,即使我从浏览器打开页面我可以看到价格,当我尝试下载带有python源代码的页面时,请求返回一个带按钮的页面:'显示价格'而不是价格。

按钮周围的页面源代码是:

data-click-store-id="sr-compset-2128695"
data-et-click="customGoal:YPNdKNKNKZJUESUPTOdJDUFYQC:1
customGoal:YPNdKNKNKZAMUVdFePOdXeRe:1"
data-et-focus="customGoal:OTfdASFOQJNDYBWfBQVT:1" target="_blank"
<span class="b-button__text"> Show prices </span>

我将此代码基于类似的question

from bs4 import BeautifulSoup

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

web_site='https://www.booking.com/searchresults.en-gb.html?aid=376363&label=bdot-1gtfXe7K0wVduEQU2KBU*QS144456159570%3Apl%3Ata%3Ap1%3Ap21%2C093%2C000%3Aac%3Aap1t1%3Aneg%3Afi%3Atiaud-146342138710%3Akwd-334108349%3Alp1008736%3Ali%3Adec%3Adm&lang=en-gb&sid=316b1ca4ddb0b74abc941811e1a769db&sb=1&src=index&src_elem=sb&error_url=https%3A%2F%2Fwww.booking.com%2Findex.en-gb.html%3Faid%3D376363%3Blabel%3Dbdot-1gtfXe7K0wVduEQU2KBU%252AQS144456159570%253Apl%253Ata%253Ap1%253Ap21%252C093%252C000%253Aac%253Aap1t1%253Aneg%253Afi%253Atiaud-146342138710%253Akwd-334108349%253Alp1008736%253Ali%253Adec%253Adm%3Bsid%3D316b1ca4ddb0b74abc941811e1a769db%3Bsb_price_type%3Dtotal%26%3B&ss=Rome%2C+Lazio%2C+Italy&ssne=Apia&ssne_untouched=Apia&checkin_monthday=28&checkin_month=10&checkin_year=2017&checkout_monthday=31&checkout_month=10&checkout_year=2017&no_rooms=1&group_adults=2&group_children=0&genius_rate=1&from_sf=1&ss_raw=rom&ac_position=0&ac_langcode=en&dest_id=-126693&dest_type=city&search_pageview_id=18384c2ba57602b5&search_selected=true&search_pageview_id=18384c2ba57602b5&ac_suggestion_list_length=5&ac_suggestion_theme_list_length=0'

driver = webdriver.PhantomJS()
driver.get(web_site)
driver.save_screenshot('screenshot1.png')
wait = WebDriverWait(driver, 30)

# click proceed
proceed = wait.until(EC.presence_of_element_located((By.LINK_TEXT, "\nShow prices\n")))

proceed.click()
# wait for the content to be present
wait.until(EC.presence_of_element_located((By.ID, "workskin")))

soup = BeautifulSoup(driver.page_source, "html.parser")
soup.prettify()

This is a saved screenshot by phantomJS

0 个答案:

没有答案