Question

如果我谷歌How old is Messi，它应该给我输出：30，但它会回答我None。
我正在使用Python 3。

import time
from bs4 import BeautifulSoup
import requests
search=input("What do you want to ask: ")
search=search.replace(" ","+")
link="https://www.google.com/search?q="+search
print(link)
source=requests.get(link).text

soup=BeautifulSoup(source,"html.parser")
print(soup.prettify())
answer=soup.find('div',class_="Z0LcW")
print(answer.text)

Answer 1

您必须添加用户代理以伪造真正的浏览器访问：

headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}
source=requests.get(link, headers=headers).text
soup=BeautifulSoup(source,"html.parser")

试过这个，它有效。有关详细信息，请参阅this answer。

Answer 2

首先你的汤。找错了，你必须这样做 answer=soup.find('div',{'class':'Z0LcW'})
最有可能的问题是你需要一些东西来加载JS数据。您可以使用selenium + ChromeDriver

import selenium
from selenium import webdriver
driver = webdriver.Chrome()
import time
from bs4 import BeautifulSoup
import requests

search=input("What do you want to ask: ")
search=search.replace(" ","+")
link="https://www.google.com/search?q="+search
driver.get(link)
time.sleep(2)
driver.implicitly_wait(5)
html = driver.page_source
soup=BeautifulSoup(html, "html.parser")
print(soup.prettify())
answer=soup.find('div',{'class':"Z0LcW")
print(answer.text)

希望这会有所帮助。 P.S：建议关注PEP-8 style guide

使用BeautifulSoup进行网页报废（Google）

2 个答案: