Question

我正在学习关于Udacity的教程，了解如何从网站上抓取数据。本教程解释了使用BeautifulSoup提取POST参数的步骤以及发送它们的请求。

from bs4 import BeautifulSoup
import requests

r = requests.get("http://www.transtats.bts.gov/Data_Elements.aspx?Data=2")
soup = BeautifulSoup(r.text)

viewstate = soup.find(id='__VIEWSTATE')["value"]
eventvalidation = soup.find(id = "__EVENTVALIDATION")["value"]

r = requests.post("http://www.transtats.bts.gov/Data_Elements.aspx?Data=2",
                 data={'AirportList' : "BOS",
                       'CarrierList' : "VX" ,
                       'Submit' : "Submit"  ,
                       '__EVENTTARGET': "",
                       '__EVENTARGUMENT': "",
                       '__EVENTVALIDATION' : eventvalidation,
                       '__VIEWSTATE' : viewstate })

有时会抛出TypeErrors来分配viewstate变量，并始终为eventvalidation变量。

这是我得到的错误：

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-24-47cdc853719c> in <module>()
----> 1 viewstate = soup.find(id='__VIEWSTATE')["value"]
      2 eventvalidation = soup.find(id = "__EVENTVALIDATION")["value"]

TypeError: 'NoneType' object has no attribute '__getitem__'

当我第一次请求网站数据并将其分配给r

时，我也得到了这个警告

/home/dude/miniconda/lib/python2.7/site-packages/bs4/dammit.py:269: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  if (len(data) >= 4) and (data[:2] == b'\xfe\xff') \
/home/dude/miniconda/lib/python2.7/site-packages/bs4/dammit.py:273: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif (len(data) >= 4) and (data[:2] == b'\xff\xfe') \
/home/dude/miniconda/lib/python2.7/site-packages/bs4/dammit.py:277: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif data[:3] == b'\xef\xbb\xbf':
/home/dude/miniconda/lib/python2.7/site-packages/bs4/dammit.py:280: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif data[:4] == b'\x00\x00\xfe\xff':
/home/dude/miniconda/lib/python2.7/site-packages/bs4/dammit.py:283: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif data[:4] == b'\xff\xfe\x00\x00':

我知道所有数据都在那里。我打电话的时候 r.text我得到了预期的输出，我可以滚动查看id =＆＃34; __ EVENTVALIDATION＆＃34;的值中的长文本字符串。在窗口。

为什么无法分配值的任何想法？

变量不会从BeautifulSoup

0 个答案: