使用python和selenium进行抓取,使元素显示文本元素

时间:2017-02-19 03:07:11

标签: python selenium web-scraping

当我上次询问此问题时,我一直在尝试使用xpath实现废料,我将展示我的代码和输出以及所需的输出:

代码:

from re import findall,sub
from lxml import html
from time import sleep
from selenium import webdriver
from pprint import pprint
from xvfbwrapper import Xvfb

def parse(url):
    response = webdriver.Chrome()

    response.get(url)

    otherkeyElement = response.find_element_by_class_name('locu-menu-item-name')

    thekeyElement = response.find_element_by_css_selector('.locu-widget-wrapper #locu-render-output .locu-menu .locu-menu-item .locu-menu-item-name')

    gelkeyElement = response.find_element_by_xpath('//*[@id="locu-medium-container"]/div[1]/div/div[1]/div[2]/div[3]/div[1]/div[1]')

    if otherkeyElement:
        otherkeyElement
        pprint(otherkeyElement)

    if thekeyElement:
        thekeyElement
        pprint(thekeyElement)

    if gelkeyElement:
        gelkeyElement
        print(gelkeyElement)
        sleep(3)

    parser = html.fromstring(response.page_source,response.current_url)

    gelato = parser.xpath('//*[@id="locu-medium-container"]/div[1]/div/div[1]/div[2]/div[2]/div[1]/div[1]')

    for gelatos in gelato:
        print(gelatos)

if __name__ == '__main__':

    parse('http://www.lunarosagelato.com/menu')

这为类名css选择器和xpath输出类似的元素:

<selenium.webdriver.remote.webelement.WebElement 
    (session="01aab8caa61f2362ef2e5d988619d742", element="0.7957563181681526-1")>

我期待的文字是牙买加Lime Sorbetto。任何帮助都会非常感激。

1 个答案:

答案 0 :(得分:2)

当您打印WebElement时,您将获得其to_string功能,就像任何其他对象一样。如果您希望与其关联的文字使用.text

otherkeyElement = response.find_element_by_class_name('locu-menu-item-name')
print(otherkeyElement.text)