Question

我正在尝试下载一些汽车VIN数据以获取个人名单。这涉及通过python3.7中的request.get向网站“ http://vin.place/search.php”发出请求。

我的代码如下：

import requests
import pandas as pd
from bs4 import BeautifulSoup

payload = {'first name':'JOHN','last name':'DOE'}
webpage_response = requests.get("http://vin.place/search.php",data = payload)

webpage = webpage_response.content
soup = BeautifulSoup(webpage,"html.parser")

results = soup.select(".search-content")

for x in results:
  print(x.get_text())

不幸的是，我发现结果= []。我怀疑我编写requests.get命令的方式有问题。我不确定该网站应该使用什么样的正确键。谁能帮忙吗？谢谢。

Answer 1

您的有效负载是错误的，应该是first，last。以及小写的名字：

import requests
from bs4 import BeautifulSoup

payload = {'first':'john','last':'doe'}
webpage_response = requests.post("http://vin.place/search.php", data=payload)

webpage = webpage_response.content
soup = BeautifulSoup(webpage,"html.parser")

results = soup.select(".search-content")

for x in results:
    print(x.text)

打印：

JOHN DOE  Purchase of a  2007 HONDA ACCORD
JOHN DOE  Purchase of a  2007 CHRYSLER TOWN AND COUNTRY
JOHN DOE  Purchase of a  2007 DODGE RAM PICKUP 2500
JOHN DOE  Purchase of a  2007 DODGE RAM PICKUP 1500
JOHN DOE  Purchase of a  2007 VOLKSWAGEN PASSAT
JOHN DOE  Purchase of a  2007 VOLKSWAGEN JETTA
JOHN DOE  Purchase of a  2007 CHRYSLER SEBRING
JOHN DOE  Purchase of a  2007 DODGE RAM PICKUP 1500
JOHN DOE  Purchase of a  2007 JEEP PATRIOT
JOHN DOE  Purchase of a  2007 Chevrolet TrailBlazer
JOHN DOE  Purchase of a  2007 FORD EDGE
JOHN DOE  Purchase of a  2007 JEEP WRANGLER UNLIMITED
JOHN DOE  Purchase of a  2007 DODGE RAM PICKUP 2500
JOHN DOE  Purchase of a  2007 FORD MUSTANG
JOHN DOE  Purchase of a  2007 JEEP WRANGLER
JOHN DOE  Purchase of a  2007 CHRYSLER 300
JOHN DOE  Purchase of a  2007 JEEP WRANGLER UNLIMITED
JOHN DOE  Purchase of a  2007 DODGE MAGNUM
JOHN DOE  Purchase of a  2007 CADILLAC ESCALADE ESV
JOHN DOE  Purchase of a  2007 FORD F-150
JOHN DOE  Purchase of a  2007 HONDA ACCORD
JOHN DOE  Purchase of a  2007 JEEP LIBERTY
JOHN DOE  Purchase of a  2007 CADILLAC ESCALADE
JOHN DOE  Purchase of a  2007 DODGE RAM PICKUP 1500
JOHN DOE  Purchase of a  2007 JEEP WRANGLER

Answer 2

手动执行搜索时，查看chrome发送的表单数据，数据为：first=JOHN&last=DOE，该网站也使用POST

请尝试以下操作：

import requests
import pandas as pd
from bs4 import BeautifulSoup

payload = {'first':'JOHN','last':'DOE'}
webpage_response = requests.post("http://vin.place/search.php",data = payload)

webpage = webpage_response.content
soup = BeautifulSoup(webpage,"html.parser")

results = soup.select(".search-content")

for x in results:
  print(x.get_text())

如何向网站提出请求并下载搜索数据

2 个答案: