Question

我具有以下功能来运行BigQuery数据提取（请参见下文）。当我发送太多请求时，我收到错误消息：

google.api_core.exceptions.Forbidden：403超出了速率限制：也此project_and_region的许多并发查询。欲了解更多信息，请参阅 https://cloud.google.com/bigquery/troubleshooting-errors

我想知道为什么我的代码无法捕获禁止错误，因为我明确编写了捕获403的函数？

from google.cloud import bigquery
from google.api_core.exceptions import Forbidden, InternalServerError, ServiceUnavailable

def run_job(query, query_params, attempt_nb=1):

    # Configure
    job_config = bigquery.QueryJobConfig()
    job_config.query_parameters = query_params
    query_job = client.query(
        query,
        # Location must match that of the dataset(s) referenced in the query.
        location='US',
        job_config=job_config)  # API request - starts the query

    # Try to run and transform to DataFrame()
    try:
        df = query_job.to_dataframe()
        assert query_job.state == 'DONE'
        return df

    except Forbidden:
        # Exception mapping a ``403 Forbidden`` response."""
        return retry_job(query, query_params, attempt_nb)

    except InternalServerError:
        # Exception mapping a ``500 Internal Server Error`` response. or a :attr:`grpc.StatusCode.INTERNAL` error."""
        return retry_job(query, query_params, attempt_nb)

    except ServiceUnavailable:
        # Exception mapping a ``503 Service Unavailable`` response or a :attr:`grpc.StatusCode.UNAVAILABLE` error."""
        return retry_job(query, query_params, attempt_nb)


def retry_job(query, query_params, attempt_nb):
    # If the error is a rate limit or connection error, wait and
    # try again.
    # 403: Forbidden: Both access denied and rate limits.
    # 408: Timeout
    # 500: Internal Service Error
    # 503: Service Unavailable
    # Old way: if err.resp.status in [403, 408, 500, 503]:
    if attempt_nb < 3:
        print(' ! New BigQuery error. Retrying in 10s')
        time.sleep(10)
        return run_job(query, query_params, attempt_nb + 1)
    else:
        raise Exception('BigQuery error. Failed 3 times', query)

Answer 1

异常很可能由以下行而不是在<BrowserRouter basename={process.env.PUBLIC_URL}> {/* other components */} </BrowserRouter>块内引发。如果发生重试并且在递归过程中再次抛出异常，则try可能是罪魁祸首。

to_dataframe()

在the source for this library处，query_job = client.query( query, # Location must match that of the dataset(s) referenced in the query. location='US', job_config=job_config) # API request - starts the query方法调用query()创建一个作业，在该作业中，根据{{3}上的Google页面检查 rateLimitExceeded }：

如果您的项目超出了并发速率限制或API请求限制（通过太快发送太多请求），则会返回此错误。

您可以通过在调用周围添加日志记录和/或将POST调用放在query()块中以查看是否能够解决问题来进一步测试。

在Python中捕获403禁止

1 个答案: