面临从建议下拉菜单中选择第二项的问题

时间:2017-08-12 18:05:35

标签: python python-3.x selenium selenium-webdriver web-scraping

我已经用硒写了一个用硒写的脚本来刮掉这个城市&使用反向查找从谷歌地图拉链。到目前为止它运作良好。我无法满足的唯一要求是:在搜索框中输入地址后,有时会出现一些建议,就像下拉菜单一样。它默认选择第一个。如果我想从建议中选择第二个,我就会陷入困境。

以下是我尝试使用的脚本:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

search_street = ['10123 HARDISON','1006 ELEANOR ST']

def find_address(driver, wait):
    for item in range(0,2):
        driver.get("https://maps.google.com/maps?q=Houston")
        wait = WebDriverWait(driver, 10)
        input_val = wait.until(EC.presence_of_element_located((By.NAME, "q")))
        input_val.clear()
        input_val.send_keys(search_street[item])
        driver.find_element_by_id('searchbox-searchbutton').click()

        # wait.until(EC.presence_of_element_located((By.XPATH, "//div[@id='sbse6']")))
        # driver.find_elements_by_xpath("//div[@id='sbse6']")[2].click()

        wait.until(EC.presence_of_element_located((By.XPATH, "//div[@class='section-hero-header-description']")))  
        element = driver.find_element_by_xpath("//div[@class='section-hero-header-description']")
        name = driver.execute_script("return arguments[0].childNodes[3].textContent", element).strip()
        print(name)

if __name__ == '__main__':
    driver = webdriver.Chrome()
    wait = WebDriverWait(driver, 10)
    try:
        find_address(driver, wait)
    finally:
        driver.quit()

建议下拉列表的Html元素:

<div class="sbdd_b" style=""><div class="suggestions"><ul class="sbsb_b" role="listbox"><li role="presentation" class="sbsb_c " dir="ltr" style="text-align: left;"><div role="option" id="sbse5"><div class="sbqs_a"></div><div class="sbqs_c"><div jstcache="746" class="suggest" jsan="7.suggest"><div jstcache="747" jsinstance="*0" class="suggest-text-layout"><div jstcache="748" class="suggest-left-cell"><div role="presentation" jstcache="749" class="suggest-icon-container maps-sprite-suggest-place-pin" jsan="7.suggest-icon-container,7.maps-sprite-suggest-place-pin,0.role"></div><img role="presentation" jstcache="750" class="suggest-activity" style="display:none"><span jstcache="751" style="display:none"><span jstcache="752" class="suggest-query" jsan="7.suggest-query"></span> nearby</span><span jstcache="753" class="suggest-query" jsan="7.suggest-query,t-fmS2vATOQZY"><span jstcache="764" jsinstance="0" class=" suggest-bold" jsan="7.suggest-bold">10123 Valley Park Dr</span><span jstcache="764" jsinstance="*1" class="">ive</span></span> <span jstcache="754"><span jstcache="764" jsinstance="*0">Houston, TX</span></span></div><div class="suggest-right-cell"><button jstcache="755" jsaction="omnibox.editAlias" class="blue-button-text suggest-edit-link" style="display:none">set location</button><button jstcache="756" jsaction="omnibox.editAlias" class="blue-button-text suggest-edit-link" style="display:none">edit</button><span jstcache="757" style="display:none"><div class="suggest-ad-creative-annotation">Ad</div><span jstcache="758"></span></span></div></div></div></div></div></li><li role="presentation" class="sbsb_c " dir="ltr" style="text-align: left;"><div role="option" id="sbse6"><div class="sbqs_a"></div><div class="sbqs_c"><div jstcache="746" class="suggest" jsan="7.suggest"><div jstcache="747" jsinstance="*0" class="suggest-text-layout"><div jstcache="748" class="suggest-left-cell"><div role="presentation" jstcache="749" class="suggest-icon-container maps-sprite-suggest-place-pin" jsan="7.suggest-icon-container,7.maps-sprite-suggest-place-pin,0.role"></div><img role="presentation" jstcache="750" class="suggest-activity" style="display:none"><span jstcache="751" style="display:none"><span jstcache="752" class="suggest-query" jsan="7.suggest-query"></span> nearby</span><span jstcache="753" class="suggest-query" jsan="7.suggest-query,t-fmS2vATOQZY"><span jstcache="764" jsinstance="0" class=" suggest-bold" jsan="7.suggest-bold">10123 Valley Park Dr</span><span jstcache="764" jsinstance="*1" class="">ive</span></span> <span jstcache="754"><span jstcache="764" jsinstance="*0">Garland, TX</span></span></div><div class="suggest-right-cell"><button jstcache="755" jsaction="omnibox.editAlias" class="blue-button-text suggest-edit-link" style="display:none">set location</button><button jstcache="756" jsaction="omnibox.editAlias" class="blue-button-text suggest-edit-link" style="display:none">edit</button><span jstcache="757" style="display:none"><div class="suggest-ad-creative-annotation">Ad</div><span jstcache="758"></span></span></div></div></div></div></div></li><li role="presentation" class="sbsb_c " dir="ltr" style="text-align: left;"><div role="option" id="sbse7"><div class="sbqs_a"></div><div class="sbqs_c"><div jstcache="746" class="suggest" jsan="7.suggest"><div jstcache="747" jsinstance="*0" class="suggest-text-layout"><div jstcache="748" class="suggest-left-cell"><div role="presentation" jstcache="749" class="suggest-icon-container maps-sprite-suggest-place-pin" jsan="7.suggest-icon-container,7.maps-sprite-suggest-place-pin,0.role"></div><img role="presentation" jstcache="750" class="suggest-activity" style="display:none"><span jstcache="751" style="display:none"><span jstcache="752" class="suggest-query" jsan="7.suggest-query"></span> nearby</span><span jstcache="753" class="suggest-query" jsan="7.suggest-query,t-fmS2vATOQZY"><span jstcache="764" jsinstance="0" class=" suggest-bold" jsan="7.suggest-bold">10123 Valley Park Dr</span><span jstcache="764" jsinstance="*1" class="">ive</span></span> <span jstcache="754"><span jstcache="764" jsinstance="*0">Garden Ridge, TX</span></span></div><div class="suggest-right-cell"><button jstcache="755" jsaction="omnibox.editAlias" class="blue-button-text suggest-edit-link" style="display:none">set location</button><button jstcache="756" jsaction="omnibox.editAlias" class="blue-button-text suggest-edit-link" style="display:none">edit</button><span jstcache="757" style="display:none"><div class="suggest-ad-creative-annotation">Ad</div><span jstcache="758"></span></span></div></div></div></div></div></li><li role="presentation" class="sbsb_c " dir="ltr" style="text-align: left;"><div role="option" id="sbse8"><div class="sbqs_a"></div><div class="sbqs_c"><div jstcache="746" class="suggest" jsan="7.suggest"><div jstcache="747" jsinstance="*0" class="suggest-text-layout"><div jstcache="748" class="suggest-left-cell"><div role="presentation" jstcache="749" class="suggest-icon-container maps-sprite-suggest-place-pin" jsan="7.suggest-icon-container,7.maps-sprite-suggest-place-pin,0.role"></div><img role="presentation" jstcache="750" class="suggest-activity" style="display:none"><span jstcache="751" style="display:none"><span jstcache="752" class="suggest-query" jsan="7.suggest-query"></span> nearby</span><span jstcache="753" class="suggest-query" jsan="7.suggest-query,t-fmS2vATOQZY"><span jstcache="764" jsinstance="0" class=" suggest-bold" jsan="7.suggest-bold">10123 Valley Park Dr</span><span jstcache="764" jsinstance="*1" class="">ive</span></span> <span jstcache="754"><span jstcache="764" jsinstance="*0">San Antonio, TX</span></span></div><div class="suggest-right-cell"><button jstcache="755" jsaction="omnibox.editAlias" class="blue-button-text suggest-edit-link" style="display:none">set location</button><button jstcache="756" jsaction="omnibox.editAlias" class="blue-button-text suggest-edit-link" style="display:none">edit</button><span jstcache="757" style="display:none"><div class="suggest-ad-creative-annotation">Ad</div><span jstcache="758"></span></span></div></div></div></div></div></li><li role="presentation" class="sbsb_c " dir="ltr" style="text-align: left;"><div role="option" id="sbse9"><div class="sbqs_a"></div><div class="sbqs_c"><div jstcache="746" class="suggest" jsan="7.suggest"><div jstcache="747" jsinstance="*0" class="suggest-text-layout"><div jstcache="748" class="suggest-left-cell"><div role="presentation" jstcache="749" class="suggest-icon-container maps-sprite-suggest-place-pin" jsan="7.suggest-icon-container,7.maps-sprite-suggest-place-pin,0.role"></div><img role="presentation" jstcache="750" class="suggest-activity" style="display:none"><span jstcache="751" style="display:none"><span jstcache="752" class="suggest-query" jsan="7.suggest-query"></span> nearby</span><span jstcache="753" class="suggest-query" jsan="7.suggest-query,t-fmS2vATOQZY"><span jstcache="764" jsinstance="0" class=" suggest-bold" jsan="7.suggest-bold">10123 Valley Park Dr</span><span jstcache="764" jsinstance="*1" class="">ive</span></span> <span jstcache="754"><span jstcache="764" jsinstance="*0">Temple, TX</span></span></div><div class="suggest-right-cell"><button jstcache="755" jsaction="omnibox.editAlias" class="blue-button-text suggest-edit-link" style="display:none">set location</button><button jstcache="756" jsaction="omnibox.editAlias" class="blue-button-text suggest-edit-link" style="display:none">edit</button><span jstcache="757" style="display:none"><div class="suggest-ad-creative-annotation">Ad</div><span jstcache="758"></span></span></div></div></div></div></div></li></ul></div></div>

我所说的建议下拉是这样的: image_link

顺便说一句,我试图从建议下拉列表中选择第二个选项的xpath也在我的脚本中给出,此脚本此时显示为灰色。

1 个答案:

答案 0 :(得分:1)

我无法用xpath做到这一点很可能是因为我发现页面变化非常快,元素变得很陈旧。但是我可以用类来完成它:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from time import sleep

search_street = ['10123 HARDISON','1006 ELEANOR ST']

def find_address(driver, wait):
    for item in range(0,2):
        driver.get("https://maps.google.com/maps?q=Houston")
        wait = WebDriverWait(driver, 10)
        input_val = wait.until(EC.presence_of_element_located((By.NAME, "q")))
        input_val.clear()
        input_val.send_keys(search_street[item])

        wait.until(EC.presence_of_element_located((By.CLASS_NAME, "suggest")))
        sleep(5) # To prevent stale element reference as the text changes.
        for element in driver.find_elements_by_class_name("suggest"):
            print (element.text)

if __name__ == '__main__':
    driver = webdriver.Chrome()
    wait = WebDriverWait(driver, 10)
    try:
        find_address(driver, wait)
    finally:
        driver.quit()

输出:

10123 Hardison Lane Houston, TX
10123 Hardison Lane Abilene, TX
10123 Hardison Road Woodburn, KY
10123 Hardison Road Charlotte, NC
10123 Hardison Road Columbia, TN
1006 Eleanor Street College Station, TX
1006 Eleanor Street Houston, TX
1006 Eleanor Street Victoria, TX
1006 Eleanor Street La Marque, TX
1006 Eleanor Street Brenham, TX

您可以对所选元素执行点击事件,如下所示:

driver.find_elements_by_class_name("suggest")[2].click()