从html selenium读取元素

时间:2015-07-06 14:00:50

标签: python file selenium selenium-webdriver web-scraping

我正在尝试使用硒记录tf2市场上的每个项目。我正在尝试记录待售文件中每个项目的名称。 This是该页面的链接。我认为这是这个标签我只是不知道如何在一个文本文件中引用和记录名称,每个名字都在新行上。

mySet.remove(mySet.first())

编辑1:

我已经使用了alecxe的解决方案,它适用于第一页我正在尝试运行它以选择下一个按钮然后再次运行。但无济于事,这就是我的努力。

<span id="result_0_name" class="market_listing_item_name" style="color; #7D6D00;">

这会产生此错误

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

from selenium import webdriver
url="http://steamcommunity.com/market/search?appid=440#p1_popular_desc"
driver = webdriver.Firefox()
driver.get(url)

x=1
while x==1:
    WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.market_listing_row")))
    time.sleep(5)
    results = [item.text for item in driver.find_elements_by_css_selector("div.market_listing_row .market_listing_item_name")]
    time.sleep(5)
    driver.find_element_by_id('searchResults_btn_next').click()
    with open("output.dat", "a") as f:
        for item in results:
            f.write(item + "\n")

即使是链接指南

,也会非常感谢任何帮助

1 个答案:

答案 0 :(得分:1)

您可以从market_listing_item_name类名称的元素中获取名称,这些元素位于具有div类的market_listing_row元素中:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

from selenium import webdriver

url = "http://steamcommunity.com/market/search?appid=440"
driver = webdriver.Chrome()
driver.get(url)

# wait for results
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.market_listing_row")))

results = [item.text for item in driver.find_elements_by_css_selector("div.market_listing_row .market_listing_item_name")]

driver.quit()

# dump results to a file
with open("output.dat", "wb") as f:
    for item in results:
        f.write(item + "\n")

以下是运行脚本后output.dat文件的内容:

Mann Co. Supply Crate Key
The Powerhouse Weapons Case
The Concealed Killer Weapons Case
Earbuds
Bill's Hat
Gun Mettle Campaign Pass
Tour of Duty Ticket
Genuine AWPer Hand
Specialized Killstreak Kit
Gun Mettle Key