我正在运行测试脚本,即使经过大量的搜索,我也收到了一个我不理解的错误。 这是与代码相关的部分,在这里我有一个循环,该循环从列表中读取并在线查找值并获取某些值并将其写入数据帧。它运行良好,并且我通过了数百个查找值,然后中断了。
在执行此操作时,我不会打开任何其他窗口,并且脚本不会搜索除打开的页面之外的任何内容。我可能会收到与以下问题类似的错误:"NoSuchWindowException: no such window: window was already closed" while switching tabs using Selenium and WebDriver through Python3 但是由于我只使用单页和单个框架,因此我没有进行相同类型的刮擦。
for i in lookup_list:
data = pd.DataFrame()
start_url = f"https://www.amazon.com/s?k=" + i +"&ref=nb_sb_noss_1"
browser.visit(start_url)
if browser.find_by_xpath('(.//span[@class = "a-size-medium a-color-base"])[1]') is not None :
item = browser.find_by_xpath("//a[@class='a-link-normal']")
try:
item.click()
html = browser.html
soup = bs(html, "html.parser")
collection_dict ={
'ERP_VENDOR_PART_NUMBER': i,
'DIMENSIONS': getdim(soup),
'IMAGE_LINK': getImage(soup),
'PRODUCT_LINK': start_url}
except AttributeError:
continue
data = data.append(collection_dict, ignore_index=True)
if not os.path.isfile("final_list.csv"):
data.to_csv("final_list.csv",header = "column_names")
else:
data.to_csv("final_list.csv", mode = "a", header = False)
这是错误:
$ python scraper.py
[11652:7792:0917/132302.774:ERROR:configuration_policy_handler_list.cc(90)] Unknown policy: AlwaysAuthorizePlugins
[11652:7792:0917/132302.775:ERROR:configuration_policy_handler_list.cc(90)] Unknown policy: DisablePluginFinder
[11652:7792:0917/132302.775:ERROR:configuration_policy_handler_list.cc(90)] Unknown policy: DisableSpdy
[11652:7792:0917/132302.775:ERROR:configuration_policy_handler_list.cc(90)] Unknown policy: DnsPrefetchingEnabled
[11652:7792:0917/132302.843:ERROR:configuration_policy_handler_list.cc(90)] Unknown policy: AlwaysAuthorizePlugins
[11652:7792:0917/132302.843:ERROR:configuration_policy_handler_list.cc(90)] Unknown policy: DisablePluginFinder
[11652:7792:0917/132302.844:ERROR:configuration_policy_handler_list.cc(90)] Unknown policy: DisableSpdy
[11652:7792:0917/132302.844:ERROR:configuration_policy_handler_list.cc(90)] Unknown policy: DnsPrefetchingEnabled
DevTools listening on ws://127.0.0.1:50668/devtools/browser/3d3942bb-c0e1-4d24-a8f5-788761d3a7bc
Traceback (most recent call last):
File "scraper.py", line 78, in <module>
if browser.find_by_xpath('(.//span[@class = "a-size-medium a-color-base"])[1]') is not None :
File "C:\Anaconda3\envs\PythonData\lib\site-packages\splinter\driver\webdriver\__init__.py", line 463, in find_by_xpath
original_query=original_query,
File "C:\Anaconda3\envs\PythonData\lib\site-packages\splinter\driver\webdriver\__init__.py", line 430, in find_by
elements = finder(selector)
File "C:\Anaconda3\envs\PythonData\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 410, in find_elements_by_xpath
return self.find_elements(by=By.XPATH, value=xpath)
File "C:\Anaconda3\envs\PythonData\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1007, in find_elements
'value': value})['value'] or []
File "C:\Anaconda3\envs\PythonData\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Anaconda3\envs\PythonData\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchWindowException: Message: no such window: window was already closed
(Session info: chrome=76.0.3809.132)
[11652:12664:0917/132411.117:ERROR:browser_process_sub_thread.cc(203)] Waited 59 ms for network service
在出现错误的情况下,我什至还添加了“ try”和异常,但仍会中断。 知道如何解决这个问题吗?