民间, 试图从内部公司网站上抓取一些不提供API的数据:(
网站上的表单包含USERNAME
,PASSWORD
,但其操作不是“登录”,而是指向CA页面的链接,即'action': '/something/forms/login.fcc'
import requests
headers = {'User-Agent': 'Mozilla/5.0'}
payload = {
'action': '/something/forms/login.fcc',
'USERNAME':'foo',
'PASSWORD':'bar'
}
s = requests.Session()
s.post('https://internal/unprotected/login.asp',headers=headers,data=payload)
x = s.get('https://internal/blah/mainSearch.cfm')
print x.text
让我回到登录界面......有什么建议吗?
感谢!!!
答案 0 :(得分:1)
我遇到了同样的问题。这就是我修复它的方法 - 看看这对你有用......
from requests import Request, Session
session = Session()
post_request = Request('POST', 'https://internal/unprotected/login.asp', headers=headers, data=payload)
prepare_post = session.prepare_request(post_request)
post_response = session.send(prepare_post)
get_request = Request('GET', 'https://internal/blah/mainSearch.cfm')
prepare_get = session.prepare_request(get_request)
get_response = session.send(prepare_get)