我正在尝试编写一个Python脚本,该脚本转到内部网站上的表单(名称为" DefaultForm")并填写输入名称="用户名"带有' user001'的表单中的字段,输入名称="密码"带有' pass001'的字段然后点击提交
尝试用Selenium做这件事并且它有效。但是想要用Requests完成相同的任务(以及稍后用于某些html抓取的BeautifulSoup)
我编写的代码不起作用!
url = 'http://SERVER:PORT/dashboard/portal'
payload = {'username':'user001','password':'pass001'}
r = requests.get(url, params=payload)
print(r.text)
我在request.get(..)之前和之后检查r.text中的内容,两者都是相同的。
任何人都可以帮我解决这个问题吗?
编辑/更新:还尝试使用lxml提交我的表单,但似乎有一个错误,我似乎无法理解
page = parse(url).getroot()
page.forms[0].fields['username'] = 'user001'
page.forms[0].fields['password'] = 'pass001'
result = parse(submit_form(page.forms[0]).encode('utf-8')).getroot()
print(result.text)
这是我得到的控制台结果
RUNFILE(' C:/Users/mgreza/Downloads/WinPython-64bit-3.5.1.3/notebooks/temp.py' ;, WDIR =' C:/Users/mgreza/Downloads/WinPython-64bit-3.5.1.3/notebooks') 回溯(最近一次调用最后一次):
文件"",第1行,in RUNFILE(' C:/Users/mgreza/Downloads/WinPython-64bit-3.5.1.3/notebooks/temp.py' ;, WDIR =' C:/Users/mgreza/Downloads/WinPython-64bit-3.5.1.3/notebooks')
文件 " C:\ Users \用户mgreza \下载\ WinPython-64-3.5.1.3 \蟒-3.5.1.amd64 \ lib中\站点包\ spyderlib \部件\ externalshell \ sitecustomize.py&#34 ;, 第841行,在runfile中 execfile(filename,namespace)
文件 " C:\ Users \用户mgreza \下载\ WinPython-64-3.5.1.3 \蟒-3.5.1.amd64 \ lib中\站点包\ spyderlib \部件\ externalshell \ sitecustomize.py&#34 ;, 第103行,在execfile中 exec(compile(f.read(),filename,' exec'),命名空间)
文件 的" C:/Users/mgreza/Downloads/WinPython-64bit-3.5.1.3/notebooks/temp.py", 第13行, result = parse(submit_form(page.forms [0])。encode(' utf-8'))。getroot()
文件 的" C:\ Users \用户mgreza \下载\ WinPython-64-3.5.1.3 \蟒-3.5.1.amd64 \ lib中\站点包\ LXML \ html__init __ PY", 第1110行,在submit_form 中 返回open_http(form.method,url,values)
文件 的" C:\ Users \用户mgreza \下载\ WinPython-64-3.5.1.3 \蟒-3.5.1.amd64 \ lib中\站点包\ LXML \ html__init __ PY", 第1131行,在open_http_urllib 中 返回urlopen(网址,数据)
文件 " C:\ Users \用户mgreza \下载\ WinPython-64-3.5.1.3 \蟒-3.5.1.amd64 \ lib中\的urllib \ request.py&#34 ;, 第162行,在urlopen中 return opener.open(url,data,timeout)
文件 " C:\ Users \用户mgreza \下载\ WinPython-64-3.5.1.3 \蟒-3.5.1.amd64 \ lib中\的urllib \ request.py&#34 ;, 第463行,公开 req = meth(req)
文件 " C:\ Users \用户mgreza \下载\ WinPython-64-3.5.1.3 \蟒-3.5.1.amd64 \ lib中\的urllib \ request.py&#34 ;, 第1170行,在do_request_中 引发TypeError(msg)
TypeError:POST数据应该是字节或可迭代的字节。它 不能是str类型。
请帮助!!!!!!!
这是我要提交的页面
<html>
<head>
<base href="http://SERVER/DIRECTORY">
<link href="css/default.css" type="text/css" rel="stylesheet">
<title>XYZ</title>
</head>
<body bgcolor="#ffffff">
<p> </p>
<table width="100%" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td>
<form method="POST" name="DefaultForm" action="http://SERVER/DIRECTORY" onsubmit="return (isReady(this));" autocomplete="off" _lpchecked="1">
<input name="action" type="hidden" value="JLoginUser">
<input name="serverTimeStamp" type="hidden" value="1467104268529">
<input name="clientTimeStamp" type="hidden" value="1467104268904">
<input name="clientIP" type="hidden" value="10.221.12.67">
<table height="400" cellspacing="0" cellpadding="0" width="540" align="center" background="images/bkground.gif" border="0">
<tbody>
<tr>
<td>
<table heigh="395" cellspacing="0" cellpadding="0" width="100%" background="images/Transparent.gif" border="0">
<tbody>
<tr>
<td>
<img height="19" src="images/Transparent.gif">
</td>
<td>
<img src="images/logo.gif" align="top" border="0">
</td>
<td>
<img height="19" src="images/Transparent.gif" width="8">
</td>
</tr>
<tr>
<td width="10">
<img height="1" src="images\Transparent.gif" width="10">
</td>
<td valign="top" width="497">
<table cellspacing="0" cellpadding="0" width="100%" background="images/Transparent.gif" border="0">
<tbody>
<tr>
<td valign="top" align="left" width="90"><span> </span>
<table cellspacing="0" cellpadding="0" width="100%" background="images/Transparent.gif" border="0">
<tbody>
<tr>
<td colspan="3"> </td>
</tr>
<tr>
<td width="4"> </td>
<td>
<img height="1" src="images/Transparent.gif" width="5">
</td>
<td><span> </span>
</td>
</tr>
<tr>
<td colspan="3"> </td>
</tr>
</tbody>
</table>
</td>
<td width="18">
<img height="1" src="images/Transparent.gif" width="18">
</td>
<td valign="top">
<table cellspacing="0" cellpadding="0" width="100%" background="images/Transparent.gif" border="0">
<tbody>
<tr>
<td colspan="2"> </td>
</tr>
<tr>
<td valign="top" colspan="2" height="15">
<p>
<img height="15" src="images/Transparent.gif" width="1">
</p>
</td>
</tr>
<tr>
<td valign="top" align="center" colspan="2">
<img src="images/Integrator_login.gif">
</td>
</tr>
<tr>
<td colspan="2">
<table cellspacing="0" cellpadding="0" width="100%" background="images/Transparent.gif" border="0">
<tbody>
<tr>
<td valign="top" align="center">
<div id="xboxLogin">
<b class="xtop"><b class="xb1"></b><b class="xb2"></b><b class="xb3"></b><b class="xb4"></b><b class="xb5"></b></b>
<div class="xboxLoginContent">
<table cellspacing="0" cellpadding="0" width="200" background="images/Transparent.gif" border="0">
<tbody>
<tr>
<td valign="bottom" align="left" width="11" height="11"></td>
<td height="11">
<img height="11" src="images/Transparent.gif" width="1">
</td>
<td height="11">
<img height="11" src="images/Transparent.gif" width="1">
</td>
<td valign="bottom" align="right" width="11" height="11"></td>
</tr>
<tr>
<td width="11"> </td>
<td class="loginForm" colspan="2">Please sign in</td>
<td width="11"> </td>
</tr>
<tr>
<td width="11"> </td>
<td class="loginForm"> </td>
<td height="11">
<img height="11" src="images/Transparent.gif" width="1">
</td>
<td width="11"> </td>
</tr>
<tr>
<td width="11"> </td>
<td class="loginForm">User ID</td>
<td height="11">
<img height="11" src="images/Transparent.gif" width="1">
</td>
<td width="11"> </td>
</tr>
<tr>
<td width="11"> </td>
<td class="loginForm" colspan="2">
<input class="inputStyle" type="Input" name="username" size="20" style="cursor: pointer; background-image: url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAASCAYAAABSO15qAAAAAXNSR0IArs4c6QAAAUBJREFUOBGVVE2ORUAQLvIS4gwzEysHkHgnkMiEc4zEJXCMNwtWTmDh3UGcYoaFhZUFCzFVnu4wIaiE+vvq6+6qTgthGH6O4/jA7x1OiCAIPwj7CoLgSXDxSjEVzAt9k01CBKdWfsFf/2WNuEwc2YqigKZpK9glAlVVwTTNbQJZlnlCkiTAZnF/mePB2biRdhwHdF2HJEmgaRrwPA+qqoI4jle5/8XkXzrCFoHg+/5ICdpm13UTho7Q9/0WnsfwiL/ouHwHrJgQR8WEwVG+oXpMPaDAkdzvd7AsC8qyhCiKJjiRnCKwbRsMw9hcQ5zv9maSBeu6hjRNYRgGFuKaCNwjkjzPoSiK1d1gDDecQobOBwswzabD/D3Np7AHOIrvNpHmPI+Kc2RZBm3bcp8wuwSIot7QQ0PznoR6wYSK0Xb/AGVLcWwc7Ng3AAAAAElFTkSuQmCC"); background-attachment: scroll; background-size: contain; background-position: 98% 50%; background-repeat: no-repeat;"
autocomplete="off">
</td>
<td width="11"> </td>
</tr>
<tr>
<td width="11" height="5"> </td>
<td height="5"> </td>
<td height="11">
<img height="11" src="images/Transparent.gif" width="1">
</td>
<td width="11" height="5"> </td>
</tr>
<tr>
<td width="11"> </td>
<td class="loginForm">Password</td>
<td height="11">
<img height="11" src="images/Transparent.gif" width="1">
</td>
<td width="11"> </td>
</tr>
<tr>
<td width="11"> </td>
<td colspan="2">
<input class="inputStyle" type="password" maxlength="28" name="password" size="20" style="cursor: auto; background-image: url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAASCAYAAABSO15qAAAAAXNSR0IArs4c6QAAAUBJREFUOBGVVE2ORUAQLvIS4gwzEysHkHgnkMiEc4zEJXCMNwtWTmDh3UGcYoaFhZUFCzFVnu4wIaiE+vvq6+6qTgthGH6O4/jA7x1OiCAIPwj7CoLgSXDxSjEVzAt9k01CBKdWfsFf/2WNuEwc2YqigKZpK9glAlVVwTTNbQJZlnlCkiTAZnF/mePB2biRdhwHdF2HJEmgaRrwPA+qqoI4jle5/8XkXzrCFoHg+/5ICdpm13UTho7Q9/0WnsfwiL/ouHwHrJgQR8WEwVG+oXpMPaDAkdzvd7AsC8qyhCiKJjiRnCKwbRsMw9hcQ5zv9maSBeu6hjRNYRgGFuKaCNwjkjzPoSiK1d1gDDecQobOBwswzabD/D3Np7AHOIrvNpHmPI+Kc2RZBm3bcp8wuwSIot7QQ0PznoR6wYSK0Xb/AGVLcWwc7Ng3AAAAAElFTkSuQmCC"); background-attachment: scroll; background-size: contain; background-position: 98% 50%; background-repeat: no-repeat;"
autocomplete="off">
</td>
<td width="11"> </td>
</tr>
<tr>
<td width="11" height="5"> </td>
<td height="5"> </td>
<td height="11">
<img height="11" src="images/Transparent.gif" width="1">
</td>
<td width="11" height="5"> </td>
</tr>
<tr>
<td width="6"> </td>
<td class="loginForm">
<input class="submit" name="submit" type="submit" value="Sign In" style="font-size:10">
</td>
<td width="50%" align="center"> </td>
<td width="11"> </td>
</tr>
<tr>
<td valign="top" align="left" width="11" height="11"></td>
<td height="11">
<img height="11" src="images/Transparent.gif" width="1">
</td>
<td height="11">
<img height="11" src="images/Transparent.gif" width="1">
</td>
<td valign="top" align="right" width="11" height="11"></td>
</tr>
</tbody>
</table>
</div>
<b class="xbottom"><b class="xb5"></b><b class="xb4"></b><b class="xb3"></b><b class="xb2"></b><b class="xb1"></b></b>
</div>
</td>
<td width="43">
<img height="8" src="images/Transparent.gif" width="43">
</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td valign="top" align="left"> </td>
<td valign="top" align="right" colspan="3">
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</form>
</td>
</tr>
</tbody>
</table>
</body>
</html>
&#13;
编辑/更新:在发布表单时记录
请求标题: -
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip, deflate
Accept-Language:en-US,en;q=0.8,en-GB;q=0.6
Cache-Control:max-age=0
Connection:keep-alive
Content-Length:140
Content-Type:application/x-www-form-urlencoded
Cookie:JSESSIONID=1ax740u3chasqa4rmen8ifq5b; SCI_DLSSO=U2Vzc2lvbklEaGl1YWFseGJqeGx1MTN1OGFtaXpva3Yybw==
Host:10.1.28.189:5010
Origin:http://SERVER:PORT
Referer:http://SERVER:PORT/dashboard/
Upgrade-Insecure-Requests:1
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36
响应标题: -
Cache-Control:no-cache
Content-Type:text/html; charset=utf-8
Expires:Thu, 01 Jan 1970 00:00:00 GMT
Last-Modified:Thu, 30 Jun 2016 01:47:48 GMT
Pragma:no-cache
Set-Cookie:JSESSIONID=17zql1r4hdylrfg54lardx14p;Path=/dashboard/;HttpOnly
Set-Cookie:SCI_DLSSO=U2Vzc2lvbklEMTd6cWwxcjRoZHlscmZnNTRsYXJkeDE0cA==;Path=/;HttpOnly
Transfer-Encoding:chunked
X-FRAME-OPTIONS:SAMEORIGIN
表单数据:
action=JLoginUser&serverTimeStamp=1467251264480&clientTimeStamp=2146&clientIP=10.220.12.101&username=user001&password=pass001&submit=Sign+In
答案 0 :(得分:1)
这应该让你登录:
import requests
from lxml import html
with requests.Session() as s:
xml = .get("http://SERVER:PORT/dashboard/portal").getroot()
form = xml.xpath("//form[@name='DefaultForm']")[0]
print(form.xpath("./input[@name]"))
data = {i.xpath("@name")[0] : i.xpath("@value")[0] for i in form.xpath("./input[@name]")}
post_url = form.xpath("@action")[0]
data["username"] = "username"
data["password"] = "password"
data["submit"] = "Sign In"
r = s.post("http://SERVER:PORT/dashboard/portal"", data=data)