在Python中获取XHR的响应内容

时间:2017-11-23 18:49:12

标签: python json error-handling web-scraping xmlhttprequest

我正在尝试从网站https://www.whoscored.com获取XHR的回复内容。这是我的代码

import requests

url = 'https://www.whoscored.com/StatisticsFeed/1/GetPlayerStatistics'
params = {
    'category': 'shots',
    'subcategory': 'situation',
    'isCurrent': 'false',
    'playerId': '17',
    'teamId': '',
    'matchId': '',
    'stageId': '',
    'tournamentOptions': '',
    'sortBy': 'seasonId',
    'sortAscending': '',
    'age': '',
    'ageComparisonType': '0',
    'appearances': '',
    'appearancesComparisonType': '0',
    'field': '',
    'nationality': '',
    'positionOptions': '^%^27FW^%^27,^%^27AML^%^27,^%^27AMC^%^27,^%^27AMR^%^27,^%^27ML^%^27,^%^27MC^%^27,^%^27MR^%^27,^%^27DMC^%^27,^%^27DL^%^27,^%^27DC^%^27,^%^27DR^%^27,^%^27GK^%^27,^%^27Sub^%^27',
    'timeOfTheGameEnd': '5',
    'timeOfTheGameStart': '0',
    'isMinApp': '',
    'page': '1',
    'includeZeroValues': 'true',
    'numberOfPlayersToPick': ''
}
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36',
        'X-Requested-With': 'XMLHttpRequest',
        'Host': 'www.whoscored.com',
        'Accept': 'application/json, text/javascript, */*; q=0.01',
        'Referer':'https://www.whoscored.com/Players/17/History',
        'Cookie': 'localeEditionShown_en=true; permutive-session=^%^7B^%^22session_id^%^22^%^3A^%^22e5386dfb-c58a-4882-b0e1-2cc2d9518982^%^22^%^2C^%^22last_updated^%^22^%^3A^%^222017-11-22T19^%^3A10^%^3A04.522Z^%^22^%^7D; visid_incap_774904=4xMirl1lRNOgrnN+Sm9S1zNx61kAAAAAREIPAAAAAACAsmaAAbBYMBjQTCqLf/D6wOVO4hdnKjIF; incap_ses_151_774904=/LX+SNRqsR8SzJi7p3YYAjKgGloAAAAApdQygw8VYBxbv/wvl7Be7A==; _gat=1; _gat_subdomainTracker=1; _ga=GA1.2.1522498341.1508602188; _gid=GA1.2.1243543827.1511694421'
        }


res = requests.get(url, params=params, headers=headers)

fix = res.json()

print(fix)

我收到以下错误:

>>> exec(open("C:/Users/CEM/Desktop/scrapy.py").read())
Traceback (most recent call last):
  File "<pyshell#12>", line 1, in <module>
    exec(open("C:/Users/CEM/Desktop/scrapy.py").read())
  File "<string>", line 26, in <module>
  File "C:\Users\Cem\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\models.py", line 892, in json
    return complexjson.loads(self.text, **kwargs)
  File "C:\Users\Cem\AppData\Local\Programs\Python\Python36-32\lib\json\__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "C:\Users\Cem\AppData\Local\Programs\Python\Python36-32\lib\json\decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\Cem\AppData\Local\Programs\Python\Python36-32\lib\json\decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

谁能告诉我问题可能是什么?

此问题实际上与之前发布的以下讨论有关:

XHR request URL says does not exist when attempting to parse it's content 1

0 个答案:

没有答案