如何在不实际打开浏览器的情况下使用Python向服务器发送URL请求（“不使用webbrowser模块”）？

时间：2011-12-26 18:44:04

标签： python browser urllib2 mechanize

我想将此URL作为请求发送给服务器，以便在我登录时更改网站上的内容。问题是，当我使用mechanize或urllib2打开URL时，它不会改变网站上的任何内容。但是，当我使用webbrowser模块时，它确实改变了网站上的内容。我想做webbrowser模块的功能，但没有打开实际的浏览器。有没有办法做到这一点？为什么机械化和urllib2不工作？

编辑：我所说的“对网站的更改”是指我将这些东西称为“分享”和“门票”，以获取我在网站上提供的信息。我的程序找到了准确的信息（如果它们是假的，它们会将你踢掉）并使用URL将其“插入”网站。

示例网址（我所有其他人都遵循此格式）：

http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp $ JspView $ SaveAction＆安培; inPlaceID = 1020634218＆安培; xxx_c_1_f_987 ​​= HTTP％3A％2F％2Fwww.yellowpages.com％2Fpittsburgh-PA％2Fmip％2Ffamily美元-STORE-1349194％3Flid％3D1349194

机械化代码：

import mechanize
br = mechanize.Browser()
url = http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194
br.open(url)

urllib2代码：

from urllib2 import urlopen
url = http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194
page = urllib2.urlopen(url)
page.read()

webbrowser代码：

import webbrowser
url = http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194
webbrowser.open(url)

编辑＃2 我刚刚尝试了这段代码：

import urllib2
import urllib

def log_in():
    url = 'https://www.locationary.com/index.jsp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction'
    values = {'inUserName' : 'me@gmail.com',
              'inUserPass' : 'myPass'}
    data = urllib.urlencode(values)
    req = urllib2.Request(url, data)
    req.add_header('Host', 'www.locationary.com')
    req.add_header('User-Agent', 'Mozilla/5.0 (Windows NT 6.1; rv:8.0) Gecko/20100101 Firefox/8.0')
    req.add_header('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8')
    req.add_header('Accept-Language', 'en-us,en;q=0.5')
    req.add_header('Accept-Encoding','gzip, deflate')
    req.add_header('Accept-Charset','ISO-8859-1,utf-8;q=0.7,*;q=0.7')
    req.add_header('Connection','keep-alive')
    req.add_header('Referer','http://www.locationary.com/')
    req.add_header('Cookie','site_version=REGULAR; __utma=47547066.1079503560.1321924193.1322707232.1324693472.36; __utmz=47547066.1321924193.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); nickname=jacob501; locaCountry=1033; locaState=1795; locaCity=Montreal; jforumUserId=1; PMS=1; TurnOFfTips=true; Locacookie=enable; __utma=47547066.1079503560.1321924193.1322707232.1324693472.36; __utmz=47547066.1321924193.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); nickname=jacob501; PMS=1; __utmb=47547066.15.10.1324693472; __utmc=47547066; JSESSIONID=DC7F5AB08264A51FBCDB836393CB16E7; PSESSIONID=28b334905ab6305f7a7fe051e83857bc280af1a9; __utmc=47547066; __utmb=47547066.15.10.1324693472; ACTION_RESULT_CODE=ACTION_RESULT_FAIL; ACTION_ERROR_TEXT=java.lang.NullPointerException')
    req.add_header('Content-Type','application/x-www-form-urlencoded')
    response = urllib2.urlopen(req)
    page = response.read()

url2 = 'http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194'

log_in()
response2 = urllib2.urlopen(url2)
page2 = response2.read()

但它不起作用。

编辑3：来自tony的代码对我不起作用。

import urllib2
import urllib
import cookielib

data = urllib.urlencode({"inUserName":"MYUSERNAMESHOULDBEHERE", "inUserPass":"MYPASSWORDSHOULDBEHERE"})
jar = cookielib.FileCookieJar("cookies")
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(jar))
request = urllib2.Request("https://www.locationary.com/index.jsp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction", data)
opener.open(request) 
url = "http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1012432546&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Fdennys-13470813%3Flid%3D13470813"
anything = opener.open(url)
anything.read()

最终编辑！ 我终于使用Tony的建议让它工作了！

这是我的最终代码：

import urllib2
import urllib
import cookielib

data = urllib.urlencode({"inUserName":"myemail@gmail.com", "inUserPass":"mypassword"})
jar = cookielib.FileCookieJar("cookies")
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(jar))
opener.addheaders.append(('User-agent', 'Mozilla/4.0'))
opener.addheaders.append( ('Referer', 'http://www.hellboundhackers.org/index.php') )
opener.addheaders.append(('Cookie','site_version=REGULAR; __utma=47547066.912030359.1322003402.1324688192.1324930160.55; __utmz=47547066.1324655802.52.13.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=cache:dr23PN5fUj4J:www.locationary.com/%20locationary; nickname=jacob501; jforumUserId=1; PMS=1; locaCountry=1033; locaState=1786; locaCity=Vancouver; JSESSIONID=A8F241E1924CE7A25FAA8C5CA6597697; PSESSIONID=5c21c44245f978b917f17982c944a9ec2b5d2df5; Locacookie=enable; __utmb=47547066.5.10.1324930160; __utmc=47547066'))
request = urllib2.Request("https://www.locationary.com/index.jsp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction", data)
response = opener.open(request) 
url = "http://www.locationary.com/"
anything = opener.open(url)
anything.read()

我所要做的就是添加一行

opener.addheaders.append(('Cookie','site_version=REGULAR; __utma=47547066.912030359.1322003402.1324688192.1324930160.55; __utmz=

等。等等（真正长的代码行，cookie）

我还添加了一个“Referer”和“User-Agent”标题以防万一。

谢谢tony !!

2 个答案:

答案 0 :(得分：1)

首先你应该用引号写出url变量：

url = "http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194"

如果您想在不打开浏览器的情况下发送请求，可以像使用urllib一样使用urllib。

如果你需要身份验证（看起来像你这样做），你应该发送身份验证请求，获取cookie（用于它的cookielib.FileCookieJar）并在opener中设置它们。然后，您将能够打开页面并发送请求。

大概你需要这样的东西：

data=urllib.urlencode({"login":"your login or whatever, "pass":"password}) # be aware you need to change "login" and "pass" to names of fields in form you have.
jar = cookielib.FileCookieJar("cookies")
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(jar))
request = urllib2.Request("url for authentication", data)
opener.open(request) # now you should be authorized and able to send any request like logged in user, using opener

url = "http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194"
anything = opener.open(url)
anything.read()

答案 1 :(得分：0)

{"manifest":{"errorTimeout":0,"succeed":true,"errorCode":0,"serverVersion":"1.0","type":"locaaccess"},"saveResult":{"message":"You don't have permissions!","placeOpenedState":0,"isSucess":false}}

我将你的urllib放入我的浏览器中。您需要首先向我认为的网站进行身份验证，然后执行此命令。我无法向您提供有关如何登录该网站的说明，但如果您转到登录页面，它可能有一个表单，您可以通过urllib2模仿网址

相关问题
最新问题