如何在响应页面中收集字段值并将其保存在列表中?

时间:2019-01-05 15:10:35

标签: python selenium selenium-webdriver web-scraping

我是Selenium的新手,并且遇到了以下问题。

我要打开Lambda Execution Environment and Available Libraries,填写表格,单击“提交”按钮,并在响应页面中收集“距离”字段的值。

这是我当前的代码。我可以填写表格,然后单击“提交”按钮。但是,我不知道如何从响应页面收集距离值并将其保存在列表中。我将需要在循环中执行此代码。因此,每个响应都应保存在列表中。

我也不希望实际模拟浏览器的打开。相反,我希望此过程在后台运行。

from   selenium import webdriver

browser = webdriver.Firefox()
browser.get("https://www.flightmanager.com/content/timedistanceform.aspx")

departure_airport = browser.find_element_by_id("ContentPlaceHolder1_txtDepartureICAO")
arrival_airport = browser.find_element_by_id("ContentPlaceHolder1_txtArrivalICAO")
submit   = browser.find_element_by_id("ContentPlaceHolder1_BtnSubmit")

departure_airport.send_keys("LEMD")
arrival_airport.send_keys("LEBL")

submit.click()

wait = WebDriverWait( browser, 5 ) 

1 个答案:

答案 0 :(得分:1)

要循环播放,首先需要创建这样的字典列表

[
  {"departure" : "LEMD", "arrival" : "LEBL"}
  {"departure" : "AAAA", "arrival" : "BBBB"}
]

从以下元素中选择距离

<td colspan="2" class="td4" align="left">
  Distance: <span class="td5">261.30 (NM) / 300.76 (MI) / 483.93 (KM)</span><br>
  Trip Time: <span class="td5">0:49 (includes 15 minute bias and air speed at 460Kts)</span><br>
</td>

使用选择器

# CSS
td[colspan="2"] span
# or Xpath
//span[contains(text(), "KM")]

以及完整的循环代码

distance_calculator = [{"departure" : "LEMD", "arrival" : "LEBL"}]

for dc in distance_calculator:
    browser.get("https://www........com/timedistanceform.aspx")

    departure_airport = browser.find_element_by_id("ContentPlaceHolder1_txtDepartureICAO")
    arrival_airport = browser.find_element_by_id("ContentPlaceHolder1_txtArrivalICAO")
    submit   = browser.find_element_by_id("ContentPlaceHolder1_BtnSubmit")

    departure_airport.send_keys(dc["departure"])
    arrival_airport.send_keys(dc["arrival"])
    submit.click()

    distance = browser.find_element_by_css_selector('td[colspan="2"] .td5')
    #distance = browser.find_element_by_xpath('//span[contains(text(), "KM")]')

    dc["distance"] = distance.text

print(distance_calculator)

结果:

[
  {
    "departure": "LEMD",
    "arrival": "LEBL",
    "distance": "261.30 (NM) / 300.76 (MI) / 483.93 (KM)"
  },
  {
    .......
  }
]

要在后台运行或无头运行,请参见this answer