我正在使用cloudwatch-exporter从CloudWatch抓取指标并将其公开在localhost:9106/metrics
中。
其配置如下:
region: us-east-1
set_timestamp: false
metrics:
- aws_namespace: AWS/CloudFront
aws_metric_name: TotalErrorRate
aws_statistics: [Average]
aws_dimensions: [DistributionId, Region]
aws_dimensions_select:
Region: [Global]
我确实可以看到获取的指标:
$> curl localhost:9106/metrics
# HELP aws_cloudfront_total_error_rate_average CloudWatch metric AWS/CloudFront TotalErrorRate Dimensions: [DistributionId, Region] Statistic: Average Unit: Percent
# TYPE aws_cloudfront_total_error_rate_average gauge
aws_cloudfront_total_error_rate_average{job="aws_cloudfront",instance="",region="Global",distribution_id="E1XXXXXX",} 26.666666666666668
aws_cloudfront_total_error_rate_average{job="aws_cloudfront",instance="",region="Global",distribution_id="EXXXXXXX",} 0.0
aws_cloudfront_total_error_rate_average{job="aws_cloudfront",instance="",region="Global",distribution_id="E38XXXXXX",} 0.0
aws_cloudfront_total_error_rate_average{job="aws_cloudfront",instance="",region="Global",distribution_id="E6XXXXXXX",} 100.0
# HELP cloudwatch_exporter_scrape_duration_seconds Time this CloudWatch scrape took, in seconds.
# TYPE cloudwatch_exporter_scrape_duration_seconds gauge
cloudwatch_exporter_scrape_duration_seconds 14.487444391
# HELP cloudwatch_exporter_scrape_error Non-zero if this scrape failed.
# TYPE cloudwatch_exporter_scrape_error gauge
cloudwatch_exporter_scrape_error 0.0
但是,Prometheus不会抓取它们,而是输出以下日志:
level=warn ts=2018-06-20T07:00:37.578384931Z caller=scrape.go:932 component="scrape manager" scrape_pool=kubernetes-service-endpoints target=http://100.106.248.21:9106/metrics msg="Error on ingesting samples that are too old or are too far into the future" num_dropped=24
level=warn ts=2018-06-20T07:01:36.821700134Z caller=scrape.go:932 component="scrape manager" scrape_pool=kubernetes-service-endpoints target=http://100.106.248.21:9106/metrics msg="Error on ingesting samples that are too old or are too far into the future" num_dropped=24
level=warn ts=2018-06-20T07:02:35.593731873Z caller=scrape.go:932 component="scrape manager" scrape_pool=kubernetes-service-endpoints target=http://100.106.248.21:9106/metrics msg="Error on ingesting samples that are too old or are too far into the future" num_dropped=24
具体地说:
msg =“在摄取太旧或太深的样本时出错 未来”
我的猜测是CloudWatch位于弗吉尼亚州,而我们的cloudwatch_exporter
和Prometheus都位于欧盟,因此时间戳之间的差异阻止了Prometheus抓取这些指标。
因此,我猜想使用Merge Request中指出的set_timestamp: false
。
但是,这不起作用。 我不是Prometheus的专业人员,可能是配置错误。我该如何进一步调查?