GET请求使用python得到错误的响应

时间:2018-08-02 09:46:03

标签: python python-3.x request urllib

我尝试在website上发出请求 在输入字段“ _regidTextBox”中,可以放置一个ID。因此,它选择了一名游泳运动员。 我看到,POST请求执行302并执行GET请求,例如,值SwimmerID = 123456。

现在,我想在Python中执行此操作,因此无需手动填写表单。 我的代码如下:

import requests
from bs4 import BeautifulSoup as soup
import urllib.request
import urllib.parse
from urllib.error import HTTPError

my_url = "http://dsvdaten.de/Modules/Results/Individual.aspx"
dsvID = "283621"
my_url_get = "http://dsvdaten.de/Modules/Results/Individual1.aspx?SwimmerID=" + dsvID
dsvDic = {"SwimmerID": dsvID}
headers = {
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
'Accept-Encoding': 'gzip,deflate',
'Accept-Language': 'de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7',
'Connection': 'keep-alive',
'Cache-Control': 'max-age=0',
'Host': 'dsvdaten.de',
'Referer': 'http://dsvdaten.de/Modules/Results/Individual.aspx?Lang=de-DE',
'Upgrade-Insecure-Requests': '1',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'
}

def get_data(url):
    payload = {"__VIEWSTATE": "", "__VIEWSTATEGENERATOR": "", "__EVENTVALIDATION": "", "SwimmerID": "283621"}
    try:
        session_req = urllib.request.urlopen(url, timeout=10)
        page_html = session_req.read()
    except ConnectionError as ConErr:
        print("Es ist ein Fehler aufgetreten!")
        print(ConErr)
    page_soup = soup(page_html, "html.parser")
    viewstate = page_soup.find("input", {"id": "__VIEWSTATE"}).get("value")
    payload["__VIEWSTATE"] = viewstate
    viewstategenerator = page_soup.find("input", {"id": "__VIEWSTATEGENERATOR"}).get("value")
    payload["__VIEWSTATEGENERATOR"] = viewstategenerator
    eventvalidation = page_soup.find("input", {"id": "__EVENTVALIDATION"}).get("value")
    payload["__EVENTVALIDATION"] = eventvalidation
    return payload


def get_open_url(my_url, dsvDic, headers):  # GET Methode
    encodedFields = urllib.parse.urlencode(dsvDic).encode("utf-8")
    req = urllib.request.Request(url=my_url, data=encodedFields, headers=headers)
    try:
        conn = urllib.request.urlopen(req)
        resp = conn.read()
        print(resp)
        conn.close()
        return resp
    except HTTPError as e:
        error = e.read()


payload = get_data(my_url)
dsvDic.update(payload)
get_open_url(my_url_get, dsvDic, headers)

我确实收到了响应(HTTP 200),但是没有正确的游泳者(看起来像不活跃的游泳者idk)。如果我在浏览器(F-12)中处理请求,则会得到相同的结果。

如果有人提供提示,链接或改进代码,那将是很好的选择。

0 个答案:

没有答案