Question

由于https://istio.io/docs/tasks/telemetry/logs/fluentd/中的“使用Fluentd登录”非常有限，因此我尝试配置一个更完整的logentry，其中包含要发送给Fluentd的IP地址（就像在此处定义的{{3 }}）。

当我这样做时，elasticsearch抱怨无效的UTF-8字符，因为IP实际上是作为字节发送的。（请参见此处：https://github.com/istio/istio/blob/84f3b8ff576125a6c3cab667bb213c65768f68a3/install/kubernetes/helm/istio/charts/mixer/templates/config.yaml#L242）。有没有一种方法可以使用istio的表达语言或可能在现有的流利插件中转换为字符串？

以下是我发送给stdout的消息的一个示例，该消息既可以通过stdio适配器在混音器中使用，也可以通过stdout插件流利地发送到stdout。如您所见，destinationIP和sourceIP字段在fluentd的输出中确实是乱码。

Mixer stdio adapter

{"level":"info","time":"2019-09-10T17:06:09.847408Z","instance":"accesslog.instance.istio-system","apiClaims":"","apiKey":"","clientTraceId":"","connection_security_policy":"none","destinationApp":"telemetry","destinationIp":"10.1.2.64","destinationName":"istio-telemetry-cb4486d94-c8b8z","destinationNamespace":"istio-system","destinationOwner":"kubernetes://apis/apps/v1/namespaces/istio-system/deployments/istio-telemetry","destinationPrincipal":"","destinationServiceHost":"istio-telemetry.istio-system.svc.cluster.local","destinationWorkload":"istio-telemetry","grpcMessage":"","grpcStatus":"0","httpAuthority":"mixer","latency":"1.586447ms","method":"POST","permissiveResponseCode":"none","permissiveResponsePolicyID":"none","protocol":"http","receivedBytes":1251,"referer":"","reporter":"destination","requestId":"5c796abc-b379-41e6-9814-c6ef2d33dec2","requestSize":873,"requestedServerName":"","responseCode":200,"responseFlags":"-","responseSize":5,"sentBytes":141,"sourceApp":"reviews","sourceIp":"10.1.1.61","sourceName":"reviews-v1-59fd8b965b-5zp9f","sourceNamespace":"default","sourceOwner":"kubernetes://apis/apps/v1/namespaces/default/deployments/reviews-v1","sourcePrincipal":"","sourceWorkload":"reviews-v1","url":"/istio.mixer.v1.Mixer/Report","userAgent":"","xForwardedFor":"10.1.1.61"}


Fluentd stdout

2019-09-10 17:06:09.000000000 +0000 accesslog.instance.istio-system: {"sourcePrincipal":"","destinationNamespace":"istio-system","permissiveResponseCode":"none","permissiveResponsePolicyID":"none","latency":"2.250387ms","apiClaims":"","destinationOwner":"kubernetes://apis/apps/v1/namespaces/istio-system/deployments/istio-telemetry","requestedServerName":"","connection_security_policy":"none","destinationName":"istio-telemetry-cb4486d94-c8b8z","requestId":"486b70d4-85e8-4c34-aa1b-7c4a83accbb7","userAgent":"","responseCode":200,"xForwardedFor":"10.1.2.83","reporter":"destination","grpcStatus":"0","grpcMessage":"","apiKey":"","severity":"Info","destinationServiceHost":"istio-telemetry.istio-system.svc.cluster.local","sourceOwner":"kubernetes://apis/apps/v1/namespaces/default/deployments/reviews-v2","referer":"","responseFlags":"-","sourceWorkload":"reviews-v2","protocol":"http","httpAuthority":"mixer","sourceIp":"\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000��\n\u0001\u0002S","destinationApp":"telemetry","sourceApp":"reviews","method":"POST","requestSize":1902,"destinationPrincipal":"","destinationIp":"\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000��\n\u0001\u0002@","receivedBytes":2280,"destinationWorkload":"istio-telemetry","sentBytes":141,"url":"/istio.mixer.v1.Mixer/Report","sourceNamespace":"default","sourceName":"reviews-v2-d6cfdb7d6-hckpr","responseSize":5,"clientTraceId":""}

我还运行了一个tcpdump捕获来检查流利的内容，然后在字符串destinationIP之后有两个字节0xC4和0x04，这意味着后面是长度为4的字节数组。 {3}}

还有其他人遇到过这个问题，或者甚至还有解决方案吗？

Answer 1

我最终写了一个流利的过滤器插件来纠正错误的条目。

require 'fluent/plugin/filter'

module Fluent::Plugin
  class IstioCorrector < Filter
    Fluent::Plugin.register_filter('istiocorrect', self)

    def configure(conf)
      super
    end

    def filter(tag, time, record)
      if (record["destinationIp"] != nil)
        record["destinationIp"] = record["destinationIp"].unpack('CCCC').join('.')
      end
      if (record["sourceIp"] != nil)
        record["sourceIp"] = record["sourceIp"].unpack('CCCC').join('.')
      end
      record
    end
  end
end

Istio流利的适配器将IP地址发送为字节

1 个答案: