如果无法加载网站(连接错误,超时错误等),如何跳过下一个网站?

时间:2017-10-09 10:25:07

标签: python selenium browser web-scraping debian

我有一些网站想要网页抓取,但有时脚本会因连接错误或超时错误问题而停止。

我怎样才能解决这个问题并继续我的脚本?

这是我的代码:

browser = webdriver.Firefox()
datatable=[]

browser.get('https://www.flightradar24.com/data/airports/ein/arrivals')
time.sleep(5)
i=0
while i<3:
    try:
        browser.find_element_by_xpath('//button[contains(text(), "CLICK_EVENT")]').click()
        time.sleep(5) #your try and except cleaned 
        i += 1
    except:
        i=3 #corrected indentation
html = browser.page_source
soup=BeautifulSoup(html,"html.parser")
table = soup.find('table', { "class" : "table table-condensed table-hover data-table m-n-t-15" })

#webscrapping part

time.sleep(5) 

os.remove("output.csv") 

browser = webdriver.Firefox()
datatable=[]

browser.get('https://www.flightradar24.com/data/airports/grz/arrivals')
time.sleep(5)
i=0
while i<3:
    try:
        browser.find_element_by_xpath('//button[contains(text(), "CLICK_EVENT")]').click()
        time.sleep(5)
        i += 1
    except:
        i=3
html = browser.page_source
soup=BeautifulSoup(html,"html.parser")
table = soup.find('table', { "class" : "table table-condensed table-hover data-table m-n-t-15" })

#webscrapping part again

time.sleep(5) 

os.remove("output.csv") 
.
.
.

我试过这个:

try:
    browser = webdriver.Firefox()
    datatable=[]

    browser.get('LINK1')
    time.sleep(5)
    i=0
    while i<3:
        try:
            browser.find_element_by_xpath('//button[contains(text(), "CLICK_EVENT")]').click()
        time.sleep(5)
        i += 1
        except:
        i=3
except:
    print("Failed to access web-page")
    continue

但这是完全错误的,而且不起作用。我可以使用什么来跳过这个问题?

0 个答案:

没有答案