如何通过scrapy请求发送cookie数据?

时间:2019-07-16 10:35:50

标签: python cookies scrapy python-requests

我一直在尝试将标头中的cookie发送到this url并抓取json响应。要发送的Cookie是Cookie:var iter = 10; function asyncFunction(i) { return new Promise(resolve => { let response = confirm("Iterations left: " + i + ". Proceed to next iteration?"); resolve(response); }); } function iterator(iter){ // Wait for the response of the asyncFunction to proceed to next iteration asyncFunction(iter).then((response)=>{ // Proceed if true if(response) { iterator(iter-1); } }); } iterator(iter);

在使用python的请求库时,我获得了所需的json数据。这是我的Python代码。

"tid=fpopRl3WSNQ1tdxH1bZTTR15LhuBRnJg;"

这是我的蜘蛛代码:

import requests
url = "https://amazon.zappos.com/mobileapi/janus/recos/get?item=9280133&limit=5&limit_0=5&limit_1=10&limit_2=10&limit_3=15&teen=4826753&widgets=detail-1,detail-2,detail-vis-sims,detail-3"
cookie = "tid=fpopRl3WSNQ1tdxH1bZTTR15LhuBRnJg;"
headers = {'cookie':cookie}
requests.get(url, headers=headers).json()

但是蜘蛛给出了这个错误。

def parse(self, response):
        cookie = "tid=fpopRl3WSNQ1tdxH1bZTTR15LhuBRnJg;"
        url = "https://amazon.zappos.com/mobileapif/janus/recos/get?item=9280133&limit=5&limit_0=5&limit_1=10&limit_2=10&limit_3=15&teen=4826753&widgets=detail-1,detail-2,detail-vis-sims,detail-3"
        request = scrapy.Request(url=url, cookies ={'cookie': cookie},callback=self.parsejson)
        yield request

    def parsejson(self, response):
        jsonresponse = json.loads(response.text)
        print(jsonresponse)

1 个答案:

答案 0 :(得分:2)

您需要对$query = 'SELECT * FROM transactions'; $orderBy = ' ORDER BY processingTime DESC'; if (isset($request->search['status'])) { $query .= ' WHERE status = '.$request->search['status']; } $query .= $orderBy; $query = DB::select($query); $query->get(); 使用$query = 'SELECT * FROM transactions'; $orderBy = ' ORDER BY processingTime DESC'; $where = [1]; if ($condition1) { $where[] = 'condition1 = '.$condition1; } if ($condition2) { $where[] = 'condition2 = '.$condition2; } $where = implode(' AND ', $where); $query .= ' WHERE '.$where; $query .= $orderBy; $query = DB::select($query); $query->get(); 格式:

key: value

更新似乎您在cookies中有错字(mobileapi f 与mobileapi)。检查更新的代码。