Question

我正在使用Django服务器调用韩国政府的天气API，以检索大约1800个位置的天气数据。

但是，此天气API多数情况下会导致超时。我试图给休息时间。例如，对于每个请求，它将休眠0.8秒，而在每30个请求后，它将休眠30秒。但是，这对于1800个请求实际上并不是很好。一方面，它是成功的。所有其他时间，它都以失败告终。

在这种情况下，我应该怎么做才能确保一小时内完成1800个请求？有什么办法可以从之前失败的确切点重新开始请求序列？

对于那些对此代码的外观感到好奇的人，我将其发布在下面：

import requests
import pytz
import json
import time

from datetime import date, datetime

from django.http import JsonResponse
from django.views import View
from api.models import Weather, Location, Address
from api.constants import (WEATHER_API_ENDPOINT, SERVICE_KEY,
                           PAGE_NO, NUM_OF_ROWS, BASE_DATE_EMPTY,
                           BASE_TIME_EMPTY, NX_EMPTY, NY_EMPTY,
                           DATA_TYPE_JSON
                           )


class WeatherCreate(View):

    def get(self, request):
        today = date.today().strftime("%Y%m%d")
        today = str(today)
        current_time = datetime.now(pytz.timezone('Asia/Seoul')).strftime('%H')
        current_time = str(current_time) + "00"
        current_time = "1100"
        print(current_time)
        if current_time not in ("0200", "0500", "0800", "1100", "1400", "1700", "2000", "2300"):
            return JsonResponse(data={"message": "DATA IS NOT AVAILABLE AT THIS TIME. TRY AGAIN LATER."}, status=406)

        # Call every single distinct address
        addresses = Address.objects.all()
        locations = [address.location for address in addresses]
        counter = 0

        for location in locations:
            if counter >= 30 and counter % 30 == 0:
                print({"MSG": "30 locations requested. Sleeping for 30 seconds...", "counter": counter})
                time.sleep(30)
            nx = str(location.grid_x)
            ny = str(location.grid_y)
            url = WEATHER_API_ENDPOINT + SERVICE_KEY + PAGE_NO +\
                  NUM_OF_ROWS + BASE_DATE_EMPTY + today +\
                  BASE_TIME_EMPTY + current_time + NX_EMPTY + nx +\
                  NY_EMPTY + ny + DATA_TYPE_JSON

            try:
                response = requests.get(url)
                result = response.json()["response"]

            except Exception as e:
                ############### I don't know how to save the failed location weather request for retries later ################
                
                continue
                
                # return JsonResponse(data={"MSG": "Session failed", "counter": counter}, status=401)
            try:
                data = result['body']
                items = data["items"]["item"]
                sky_value = None
                am_temp = None
                pm_temp = None

                for item in items:
                    for key, value in item.items():
                        if value == "SKY" and sky_value == None:
                            sky_value = item["fcstValue"]
                        if value == "PTY" and sky_value == None:
                            sky_value = item["fcstValue"]
                        elif value == "TMN" and am_temp == None:
                            am_temp = item["fcstValue"]
                        elif value == "TMX" and pm_temp == None:
                            pm_temp = item["fcstValue"]
                sky_value = int(float(sky_value)) if sky_value != None else None
                am_temp = int(float(am_temp)) if am_temp != None else None
                pm_temp = int(float(pm_temp)) if pm_temp != None else None

                weather = Weather.objects.create(
                    location=location,
                    state=sky_value,
                    am_temp=am_temp,
                    pm_temp=pm_temp
                )
                counter += 1
                print({"message": "WEATHER GENERATED!", "counter": counter})
                time.sleep(0.3)

            except Exception as e:
                data = {"message": "ERROR: No response body"}
                print(data)
                continue

        return JsonResponse({"MSG": "FINISHED!"}, status=200)

Answer 1

尝试在单个请求上使用while循环，

while True:
    try:
        response = requests.get(url)
        result = response.json()["response"]
        break
    except Exception as e:
        print(str(e))

这样，每当请求失败时，它只会重试该请求。但是您必须确保该请求最终将成功，否则它将不断循环。您可以添加一个trial计数器来计算试验次数，如果试验次数超过5次（例如5次）以防止无限循环，则中断该计数。

调用随机失败的外部API

1 个答案: