Selenium驱动程序下载的文件被截断了

时间:2015-10-20 08:25:24

标签: python selenium selenium-webdriver

我试图使用带有selenium-webdriver的python脚本下载网页,但它一直抛出一个valueError异常,导致下载页面被截断。

当网页上有一些字符(如逗号,连字符......)时,文件似乎被截断。

代码:

    from pip.cmdoptions import global_options
    from selenium import webdriver
    from pyvirtualdisplay import Display
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC

    def contactbrowser(httppath,iterater):
        display = Display(visible=0, size=(800, 600))
        display.start()
        driver = webdriver.Firefox()#firefox_profile=fp)
        wd=driver.get(httppath)
        driver.maximize_window()
        try:
            element = WebDriverWait(driver, 10).until(
                EC.presence_of_element_located((By.TAG_NAME, "html"))
            )
            ele=driver.find_element_by_tag_name("h1")
            header1=ele.get_attribute("innerHTML")
            fullpath1=header1
            file = open("1/"+fullpath1+".html", "w")
            for ss in driver.page_source:
                file.write(bytearray([ord(ss)]))
            file.close()
            driver.close()

        except ValueError:
            print "Value error", httppath
            driver.close()
        except TypeError:
            driver.close()
        except:
            driver.close()

    list= []
    fileloc = open("file.txt", "r")
    line = fileloc.readline()
    while line:
        list.append(line)
        line = fileloc.readline()
    fileloc.close()
    count=0
    i=0
    while count<list.__len__():
        contactbrowser(list[count],i)
        count=count+1
        i=i+1

例如:下载this page会导致截断文件。

Image

编辑:当它遇到没有相应ASCII的值时会发生问题。在前面的例子中,单词&#34; first&#34;写作&#34; fi RST&#34;在文本的某处,导致下载中断,导致截断文件。

0 个答案:

没有答案