如何将Web抓取数据从selenium保存到.txt文件

时间:2018-06-14 18:12:30

标签: python selenium

我是Python的新手,刚刚完成了“使用python自动化无聊的东西”课程。我有一个脚本适用于访问网站,获取我需要的所有必要数据,并将其打印到我的控制台。我在如何实际保存/将数据导出到文件方面遇到了问题。现在我希望能够将它导出到.txt或.csv文件。感谢任何帮助,因为我无法在网上找到直截了当的答案。我只需要最后一步来完成我的项目,谢谢!

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
browser = webdriver.Chrome()

def getTen():

# Opens the browser and navigates to the page
browser.get('http://taxsales.lgbs.com/map?lat=29.437693458470175&lon=-98.4618145&zoom=9&offset=0&ordering=sale_date,street_name,address_full,uid&sale_type=SALE,RESALE,STRUCK%20OFF,FUTURE%20SALE&county=BEXAR%20COUNTY&state=TX&in_bbox=-99.516502,28.71637426279382,-97.407127,30.153924134433552')

# Waits until the page is loaded then clicks the accept button on the popup window
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[2]/div/div/div[2]/button[1]"))).click()



    # Loops through 10 times, changing the listing number to correspond with 1
for i in range(1,11):

    clickable = "/html/body/ui-view/div[2]/main/aside/div[2]/property-listing[" + str(i) + "]/article/div/a"

        # Waits until the page is loaded then clicks the view more details button on the first result
    WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, clickable))).click()

# Waits until the page is loaded, then pulls all the info from the top section of the page
    info = WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/ui-view/div/main/div/div/div[2]/property-detail-info/dl[1]")))

    # prints info to the console
    print(info.text);

    # goes back a page and repeats the process for the next listing
    browser.back()

getTen()

2 个答案:

答案 0 :(得分:0)

如果您要保存info.text,则只需打开本地文件并写入即可。例如:

with open('output.txt', 'w') as f:
    f.write(info.text)

有关读取和写入文件的更多信息,请访问:Reading and Writing Files in Python

答案 1 :(得分:-1)

您可以简单地在控制台中使用输出/获取文本,并使用以下代码在in.txt中获取文本 output.txt将是您要保存的文件名,element.text将是您要保存在.txt中的元素或文本

以open('output.txt','w')为f:

f.write(elements.text)