Question

我可以成功发送Post请求并通过Postman接收所有数据。返回此导出的python代码

import requests

url = "https://cl.ingrammicro.com/_layouts/CommerceServer/IM/SearchService.svc/Search"

payload = "{\"request\":{\"State\":\"PNavDS=N:0\",\"Keywords\":[],\"Page\":0,\"PageLayout\":0,\"Mode\":12,\"Term\":\"0\",\"ExchangeRate\":null}}"
headers = {
    'origin': "https://cl.ingrammicro.com",
    'user-agent': "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36",
    'content-type': "application/json",
    'accept': "text/plain, */*; q=0.01",
    'cache-control': "no-cache",
    'postman-token': "e4a3a94a-fa02-5ea3-2386-295a9fe21f3b"
    }

response = requests.request("POST", url, data=payload, headers=headers)

print(response.text)

此帖子请求返回整齐的 json dict。

当我在scrapy中使用相同时，我得到 500 错误。这是我的Scrapy代码。

url = "https://cl.ingrammicro.com/_layouts/CommerceServer/IM/SearchService.svc/Search"
            headers = {
                    'origin': "https://cl.ingrammicro.com",
                    'user-agent': "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36",
                    'content-type': "application/json",
                    'accept': "text/plain, */*; q=0.01",
                    'cache-control': "no-cache",

                    }

            payload = {"request":{"State":"PNavDS=N:0","Keywords":[],"Page":0,"PageLayout":0,"Mode":12,"Term":"0","ExchangeRate":'null'}}

            return FormRequest(url, method = "POST", body=json.dumps(payload),headers = headers, callback=self.parse_data)

Scrapy错误。

2017-07-31 16:59:35 [scrapy.downloadermiddlewares.retry] DEBUG: Gave up retrying <POST https://cl.i
ngrammicro.com/_layouts/CommerceServer/IM/SearchService.svc/Search> (failed 3 times): 500 Internal
Server Error
2017-07-31 16:59:35 [scrapy.core.engine] DEBUG: Crawled (500) <POST https://cl.ingrammicro.com/_lay
outs/CommerceServer/IM/SearchService.svc/Search> (referer: None)

我也尝试将其作为字符串传递（如Postman代码所示），但错误是相同的）

url = "https://cl.ingrammicro.com/_layouts/CommerceServer/IM/SearchService.svc/Search"
            headers = {
                    'origin': "https://cl.ingrammicro.com",
                    'user-agent': "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36",
                    'content-type': "application/json",
                    'accept': "text/plain, */*; q=0.01",
                    'cache-control': "no-cache",

                    }
            payload = "{\"request\":{\"State\":\"PNavDS=N:0\",\"Keywords\":[],\"Page\":0,\"PageLayout\":0,\"Mode\":12,\"Term\":\"0\",\"ExchangeRate\":null}}"
            q = {"request":{"State":"PNavDS=N:0","Keywords":[],"Page":0,"PageLayout":0,"Mode":12,"Term":"0","ExchangeRate":"null"}}
            p={}
            p['request'] = str(q['request'])
            p['request']= p['request'].replace("'",'"')



            return FormRequest(url, method = "POST", body=urlencode(q) ,headers = headers, callback=self.parse_data)

我哪里错了？

Answer 1

我所做的是用其他部分替换部分代码。但是正确的解决方案是使用json.dumps(payload)，它将为我们提供所需的确切信息，并允许按预期操作dict。附言感谢这个答案-by @zeekay

发送Post请求后scrapy抛出错误500

1 个答案: