我正在尝试使用硒进行Web抓取,但是我无法在浏览器中加载网站,这是我的初始代码:
import time
import os,sys
reload(sys)
sys.setdefaultencoding('utf8')
import re
from selenium import webdriver
from scrapy.http import TextResponse
url = 'https://www.nytimes.com/'
driver_1 = webdriver.Chrome()
driver_1.get(url)
该代码出现错误:
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH.
对应于此行:
driver_1 = webdriver.Chrome()
因此,我尝试了whereis chromium-browser
命令并获得了以下路径:
/usr/bin/chromium-browser
/etc/chromium-browser
/usr/lib/chromium-browser
/usr/bin/X11/chromium-browser
/usr/share/chromium-browser
/usr/share/man/man1/chromium-browser.1.gz
但是对于每个人,我都会遇到错误,对于这个:
driver_1 = webdriver.Chrome("/usr/bin/chromium-browser")
我得到了错误:
selenium.common.exceptions.WebDriverException: Message: Service /usr/bin/chromium-browser unexpectedly exited. Status code was: 0
我正在使用python 2.7X
编辑:
我在正确的路径here中找到了答案:
driver = webdriver.Chrome("/usr/lib/chromium-browser/chromedriver")
为此,需要运行以下命令:
sudo apt-get install chromium-chromedriver