无法使用服务超过Stackdriver Deadline

时间:2018-10-30 13:32:07

标签: python logging kubernetes stackdriver google-cloud-stackdriver

我正在使用Google Cloud Stackdriver,使用类似于以下代码段从GKE登录:

import google.cloud.logging
logging_client = google.cloud.logging.Client()
# connects the logger to the root logging handler
cloud_logger = logging_client.logger('email_adaptor')

struct = {'message':'Processed Alarm', 'valid': True}
cloud_logger.log_struct(struct, severity='INFO')

我验证了页面https://cloud.google.com/appengine/articles/deadlineexceedederrors来处理DeadlineExceededErrors,但尚不清楚是什么原因导致google.api_core.exceptions.ServiceUnavailable

过去几天来https://status.cloud.google.com/上提供了Stackdriver的状态。我每小时大约收到12次此类错误。

Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 533, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "Deadline Exceeded"
        debug_error_string = "{"created":"@1540904982.397797031","description":"Deadline Exceeded","file":"src/core/ext/filters/deadline/deadline_filter.cc","file_line":68,"grpc_status":14}"

>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/alarm-handler/helpers/email_utils.py", line 144, in forward_alarm_smtp
    severity='INFO', production=not test_mode)
  File "/home/alarm-handler/helpers/log.py", line 84, in log
    cloud_logger.log_struct(cloud_struct, severity=severity)
  File "/usr/local/lib/python3.6/dist-packages/google/cloud/logging/logger.py", line 336, in log_struct
    client.logging_api.write_entries([entry_resource])
  File "/usr/local/lib/python3.6/dist-packages/google/cloud/logging/_gapic.py", line 124, in write_entries
    partial_success=partial_success)
  File "/usr/local/lib/python3.6/dist-packages/google/cloud/logging_v2/gapic/logging_service_v2_client.py", line 341, in write_log_entries
    request, retry=retry, timeout=timeout, metadata=metadata)
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/gapic_v1/method.py", line 139, in __call__
    return wrapped_func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/retry.py", line 260, in retry_wrapped_func
    on_error=on_error,
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/retry.py", line 177, in retry_target
    return target()
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/timeout.py", line 206, in func_with_timeout
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/grpc_helpers.py", line 61, in error_remapped_callable
    six.raise_from(exceptions.from_grpc_error(exc), exc)
  File "<string>", line 3, in raise_from
google.api_core.exceptions.ServiceUnavailable: 503 Deadline Exceeded

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 533, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "Deadline Exceeded"
        debug_error_string = "{"created":"@1540905002.399793138","description":"Deadline Exceeded","file":"src/core/ext/filters/deadline/deadline_filter.cc","file_line":68,"grpc_status":14}"

过去有人遇到过这样的错误吗?您认为什么会导致此错误? 我一直在想这是否与日志记录量有关?

0 个答案:

没有答案