我正在尝试对需要登录的网站进行Web Scrap。我已经成功登录该网站,但是当我访问页面时,html与实际的html不同。请看一下我的代码:
import requests
from bs4 import BeautifulSoup
# create a session
session = requests.Session()
datas=(***a dictionary that works fine. Log in information is stored here)
session.post('https://www.example.com/Default.aspx?
ReturnUrl=%2faces%2findex.aspx',
data=datas)
r = session.get('http://www.example.com/parts.aspx?
pn=45')
print(r.url)
>>>>'http://www.example.com/parts.aspx?
pn=45'
print(r.status_code)
>>>>200
page_soup = BeautifulSoup(r.text,'html.parser')
containers=page_soup.findAll('div',{'class':'partbox'})
print(containers)
>>>>[]
当我将page_soup和r.text中的html与站点上的实际html进行比较时,它们缺少一些关键功能。有什么建议吗?