让Selenium对多个URL执行相同的过程

时间:2019-05-10 16:13:00

标签: python python-3.x selenium selenium-webdriver

我创建了一个脚本,该脚本转到房地产拍卖网站,该网站监视价格并向我发出任何更改的警报。我最初是一次监视1个URL。现在,我需要它来监视超过1个网址,并同时执行相同的确切过程。

我尝试循环浏览器中的URL,但这似乎对我不起作用(idk,如果我写错了,大概可以)。但是我需要它来打开大量新标签并转到url,或者用另一种方式来监视所有内容。

用于1个属性url的原始脚本可以很好地工作,并且它与下面的代码之间的唯一区别是browser.get

PS:如果运行脚本,并且拍卖网址已过期(这意味着它无法查看价格),则会出现错误

import time
import requests
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
import ssl
from twilio.rest import Client
from twilio.rest import TwilioRestClient
from bs4 import BeautifulSoup as soup

urls = ['https://www.hubzu.com/property/90089436968-730-Trillium-Ln-Lilburn-GA-30047',
'https://www.hubzu.com/property/90016097522-225-Marriott-Ave-Schenectady-NY-12304',
'https://www.hubzu.com/property/90016098285-17-Spring-Meadows-Dr-Ormond-Beach-FL-32174'
]

browser = webdriver.Chrome()

# Start URL switch tryouts.

while True:
    for url in urls:
        browser.get((url))


       #This is the process i need all urls to do.
        time.sleep(2)
        address = soup(browser.page_source, 'html.parser').find('span', {'class':'h1'}).text
        propertyprice = browser.find_element_by_css_selector('span.current-bid')
        currentBidText = propertyprice.text
        try:                                
            WebDriverWait(browser, 90000).until_not(
                EC.text_to_be_present_in_element((By.CSS_SELECTOR, 'span.current-bid'), currentBidText)
                )
        finally:
            print("+++ Send notifications.")
            account_sid = "***"
            auth_token = "***"
            client = Client(account_sid, auth_token)

            PhoneNumber1 = "+***"
            PhoneNumber2 = "+***"
            print("+ Send notifications to: ", PhoneNumber1, " and ", PhoneNumber2)

            sendTo1 = "{\"binding_type\":\"sms\",\"address\":\"" + PhoneNumber1 + "\"}"
            print("+ sendTo1: ", sendTo1)
            sendTo2 = "{\"binding_type\":\"sms\",\"address\":\"" + PhoneNumber2 + "\"}"
            print("+ sendTo2: ", sendTo2)

            notify_service_sid = "***"
            notification = client.notify.services(notify_service_sid).notifications.create(
                    body='There has been a change at: '+address,
                    to_binding=[sendTo1, sendTo2]
                )

            print("+ Notification SID: ", notification.sid)

            print("+++ Exit.")
    continue

1 个答案:

答案 0 :(得分:1)

您的代码似乎很好,并为我循环遍历它们。不幸的是,所有财产都不出售,因此它们都会给我带来错误,并且我无法产生预期的行为。我认为您应该实现field :posts, type: :posts_by_state do resolve fn _, _, _ -> {:ok, []} end end try:,无论发生了什么,except:都会运行,并且我无法想象您希望每隔2秒XD更新一次。最后,您也不需要finally:,它什么也没做。代码

continue

循环浏览所有属性,对我来说,示例输出是:

import time
import requests
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
import ssl
from twilio.rest import Client
from twilio.rest import TwilioRestClient
from bs4 import BeautifulSoup as soup

urls = ['https://www.hubzu.com/property/90089436968-730-Trillium-Ln-Lilburn-GA-30047',
        'https://www.hubzu.com/property/90016097522-225-Marriott-Ave-Schenectady-NY-12304',
        'https://www.hubzu.com/property/90016098285-17-Spring-Meadows-Dr-Ormond-Beach-FL-32174']

browser = webdriver.Chrome()

# Start URL switch tryouts.

while True:
    for url in urls:
        browser.get(url)

        # This is the process i need all urls to do.
        time.sleep(2)
        address = soup(browser.page_source, 'html.parser').find('span', {'class': 'h1'}).text
        try:
            propertyprice = browser.find_element_by_css_selector('span.current-bid')
            currentBidText = propertyprice.text
            WebDriverWait(browser, 90000).until(
                EC.text_to_be_present_in_element((By.CSS_SELECTOR, 'span.current-bid'), currentBidText)
                )
        except Exception as e:
            print("an error occurred on property {}:\n{}".format(address.strip(), e))
        finally:
            pass
            # print("+++ Send notifications.")
            # account_sid = "***"
            # auth_token = "***"
            # client = Client(account_sid, auth_token)
            #
            # PhoneNumber1 = "+***"
            # PhoneNumber2 = "+***"
            # print("+ Send notifications to: ", PhoneNumber1, " and ", PhoneNumber2)
            #
            # sendTo1 = "{\"binding_type\":\"sms\",\"address\":\"" + PhoneNumber1 + "\"}"
            # print("+ sendTo1: ", sendTo1)
            # sendTo2 = "{\"binding_type\":\"sms\",\"address\":\"" + PhoneNumber2 + "\"}"
            # print("+ sendTo2: ", sendTo2)
            #
            # notify_service_sid = "***"
            # notification = client.notify.services(notify_service_sid).notifications.create(
            #         body='There has been a change at: '+address,
            #         to_binding=[sendTo1, sendTo2]
            #     )
            #
            # print("+ Notification SID: ", notification.sid)
            #
            # print("+++ Exit.")

以此类推,无限循环...