使用请求库废弃具有隐藏参数的网站

时间:2018-01-12 23:33:10

标签: python login web-scraping python-requests

因此,我尝试废弃受登录页面保护的以下webpage。尽管如此,当我运行代码时,它仍然会将我重定向到登录页面。我想这可能是因为登录页面有一个隐藏的参数,虽然我真的不知道如何将它集成到我的有效载荷中。以下是我写的代码。

import requests

payloads = {"login": "pythonusername3" ,
"password": "pythonpassword3" }

with requests.Session() as session:
    session.post('https://member.expireddomains.net/login/', data=payloads)
    get = session.get("https://member.expireddomains.net/domains/pendingdelete/?o=statustld_registered&r=d").text
    print(get)

1 个答案:

答案 0 :(得分:0)

我使用PhantomJS网页驱动程序来点击登录表单,然后将cookie发送到请求模块。

driver = webdriver.PhantomJS(executable_path = r'./phantomjs')
site = 'https://www.wanikani.com/login'

print 'Opening ' + site
driver.get(site)

driver.find_element_by_id( “USER_LOGIN”)。send_keys( “usernamexxxx”)

driver.find_element_by_id( “USER_PASSWORD”)。send_keys( “passwordxxx”)     driver.find_element_by_class_name( “链接”)。单击()

cookie = driver.get_cookies()
s = requests.Session()

c = [s.cookies.set(c['name'], c['value']) for c in cookie]

https://community.wanikani.com/t/userscript-burn-manager-review-resurrect-retire/13001/56?u=polv