为GKE中的容器使用“ monitoring_v3”创建自定义指标时出错

时间:2018-11-28 16:07:41

标签: google-cloud-platform google-kubernetes-engine google-cloud-monitoring

我们收到以下错误

  

“无法编写一个或多个TimeSeries:一个或多个要点   写入频率比配置的最大采样周期更频繁   度量标准:timeSeries [0]“

当我们尝试在GKE中创建容器级别指标时。我们正在创建自定义指标,以每秒更新app_queue的长度。在正常情况下它运作良好。但是,当pod的大小超过100时,我们会在某些API调用中收到此错误,并且错误的数量非常大。

问题: 我们缺少任何配额或设置吗?或创建自定义指标的一些最佳做法?

堆栈跟踪:

      File "/usr/local/lib/python3.6/dist-packages/Base-1.0-py3.6.egg/base/custom_metrics.py", line 63, in create_custom_metric
        client.create_time_series(project_name, [series])
      File "/usr/local/lib/python3.6/dist-packages/google/cloud/monitoring_v3/gapic/metric_service_client.py", line 775, in create_time_series
        request, retry=retry, timeout=timeout, metadata=metadata)
      File "/usr/local/lib/python3.6/dist-packages/google/api_core/gapic_v1/method.py", line 139, in __call__
        return wrapped_func(*args, **kwargs)
      File "/usr/local/lib/python3.6/dist-packages/google/api_core/retry.py", line 260, in retry_wrapped_func
        on_error=on_error,
      File "/usr/local/lib/python3.6/dist-packages/google/api_core/retry.py", line 177, in retry_target
        return target()
      File "/usr/local/lib/python3.6/dist-packages/google/api_core/timeout.py", line 206, in func_with_timeout
        return func(*args, **kwargs)
      File "/usr/local/lib/python3.6/dist-packages/google/api_core/grpc_helpers.py", line 61, in error_remapped_callable
        six.raise_from(exceptions.from_grpc_error(exc), exc)
      File "<string>", line 3, in raise_from
    google.api_core.exceptions.InvalidArgument: 400 One or more TimeSeries could not be written: One or more points were written more frequently than the maximum sampling period configured for the metric.: timeSeries[0]

1 个答案:

答案 0 :(得分:1)

如果您达到任何配额限制,则应该在Stackdriver Logging中记录一个错误。您也可以访问配额页面,查看是否达到了任何限制[Menu> IAM&admin> Quotas]。

关于要遵循的规则,请查看此documentation,以获取有关如何将数据写入自定义指标类型的更多信息。将多个数据点添加到同一时间序列的最佳做法是,添加速度不要超过每分钟一次。