我在youtube视频列表中使用BeautifulSoup抓取内容。 但缩略图由JS加载。如果我只使用BeautifulSoup,我就无法得到它们。
我发现有些文章告诉我,selenium可以做到这一点。
我安装了chromedriver
root@localhost:~# chromedriver -v
ChromeDriver 2.33.506092 (733a02544d189eeb751fe0d7ddca79a0ee28cce4)
这是我的python代码
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
browser = webdriver.Chrome()
browser.get("https://www.youtube.com/results?search_query=xxx")
time.sleep(1)
elem = browser.find_element_by_tag_name("body")
no_of_pagedowns = 20
while no_of_pagedowns:
elem.send_keys(Keys.PAGE_DOWN)
time.sleep(0.2)
no_of_pagedowns -= 1
post_elems = browser.find_elements_by_class_name("branded-page-box")
for post in post_elems:
print post.text
然后我收到了错误
Traceback (most recent call last):
File "youtubelist.py", line 68, in <module>
browser = webdriver.Chrome('/usr/local/bin/chromedriver')
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
desired_capabilities=desired_capabilities)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 151, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 240, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 308, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot find Chrome binary
(Driver info: chromedriver=2.33.506092 (733a02544d189eeb751fe0d7ddca79a0ee28cce4),platform=Linux 4.9.36-x86_64-linode85 x86_64)
有谁知道如何解决此错误?谢谢!