如何从API响应中捕获所有失败的节点

时间:2019-04-16 02:36:39

标签: python-3.x web-scraping

对于每个提出的请求,都会给出响应。我的目标是捕获所有返回false的节点。目前,我只能存储最近的回复。

以下是响应失败的示例:

<?xml version="1.0" encoding="utf-8"?>
<lead_return_response xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://lead.com/api/2/">
  <success>false</success>
  <message>Lead is Already in Return List</message>
</lead_return_response>

这是我的代码块:

import requests
import json
import csv
from bs4 import BeautifulSoup


def return_leads():
    with open('return_leads.csv', 'r') as csv_file:  #opens csv file as var csv_file
        csv_reader = csv.DictReader(csv_file)  #csv.Dictreader turns values from csv 'returns_lead.csv' and converts to key values pairs

        for data in csv_reader:  #looping through data in csv file
            url = 'https://api.com.'  #request URL for API
            params={"api_key": "#########",  #api key is only static value
                    "lead_id": data['Lead ID'],  #passing dynamic values from csv to required API parameters
                    "vertical_id": data['Vertical ID'],
                    "buyer_contract_id": data['Buyer Contract ID'],
                    "return_type": data['Return Type'],
                    "return_reason_id": data['Return Reason ID'],
                    "reject_reason_id": data['Reject Reason ID']}
            req = requests.get(url, params=params)  #request is generated for each line item in the csv file
            print(req.text) #response from API request is captured

#Print Output:

<?xml version="1.0" encoding="utf-8"?>
<lead_return_response xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://cakemarketing.com/api/2/">
  <success>false</success>
  <message>Lead is Already in Return List</message>
</lead_return_response>

<?xml version="1.0" encoding="utf-8"?>
<lead_return_response xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://cakemarketing.com/api/2/">
  <success>false</success>
  <message>Lead is Already in Return List</message>
</lead_return_response>


<?xml version="1.0" encoding="utf-8"?>
<lead_return_response xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://cakemarketing.com/api/2/">
  <success>false</success>
  <message>Lead is Already in Return List</message>
</lead_return_response>




            print(url, params) #check how params are being populated with dynamic data

#Print Output:

https://affiliates.ahs.com/api/2/track.asmx/ReturnLead {'api_key': '', 'lead_id': 'CB220BCB', 'vertical_id': '43', 'buyer_contract_id': '4', 'return_type': 'submit_for_approval', 'return_reason_id': '1', 'reject_reason_id': '0'}

https://affiliates.ahs.com/api/2/track.asmx/ReturnLead {'api_key': '', 'lead_id': 'D8E43B8D', 'vertical_id': '43', 'buyer_contract_id': '4', 'return_type': 'submit_for_approval', 'return_reason_id': '1', 'reject_reason_id': '0'}

https://affiliates.ahs.com/api/2/track.asmx/ReturnLead {'api_key': '', 'lead_id': '9B45E52E', 'vertical_id': '43', 'buyer_contract_id': '4', 'return_type': 'submit_for_approval', 'return_reason_id': '1', 'reject_reason_id': '0'}

            response = BeautifulSoup(req.text, 'lxml')
            print('response:', response)
            responseFalse = response.find('success').string
            responseMessage = response.find('message').string
            if responseFalse == 'false':
                captureResponse = []
                captureResponse.append(data['Lead ID'])
                captureResponse.append(responseFalse)
                captureResponse.append(responseMessage)

print('caputureResponse:', captureResponse)

#Print Output:

 ['9B45E52E', 'false', 'Lead is Already in Return List']

return_leads()

我的目标是使captureResponse []包含所有失败的3条线索,而不是仅包含最后一条。有人可以纠正我在这里做错的事情吗?预先谢谢你!

1 个答案:

答案 0 :(得分:0)

只需初始化一次

请初始化函数顶部附近的列表:

def return_leads():
    captureResponse = []
    with open( ...

您的代码正在该函数中进行初始化, 在for循环中, 在测试中是否为假。

结果是将一个非常好的值#1放入列表中, 然后将列表设置为[],然后在其中添加一个非常好的值#2。 并在值3进入之前再次将其重置为[]

数据结构

另外,三个.append()调用可能不是您想要的。 构造一个元组并附加 that

会更有意义。
            captureResponse.append((data['Lead ID'], responseFalse, responseMessage))

在单独的行中拆分时,元组可能会更清晰:

            summary = data['Lead ID'], responseFalse, responseMessage
            captureResponse.append(summary)