Question

我在appengine上运行了一个java应用程序。

我正在以json结构记录我的日志，然后我可以在stack driver上看到我的日志（如docs中所示）

package com.foo.bar;

public class MyClass {

    private static final Logger log = Logger.getLogger(MyClass.class.getName());

    public void myFunc() {
        log.info("{msg: 'hello', corId: '123'}");
    }

这是我在stackdriver-logging上得到的消息：

com.foo.bar.MyClass myFunc: {msg: 'hello', corId: '123'}

并在日志请求对象中：

protoPayload.line[].logMessage = "com.foo.bar.MyClass myFunc: {msg: 'hello', corId: '123'}"

如何使日志消息只是我正在记录的消息 - 没有类前缀： {msg: 'hello', corId: '123'}

protoPayload.line[].logMessage = "{msg: 'hello', corId: '123'}"

Answer 1

我最终通过logstash将日志从stackdriver发送到alasticsearch

在logstash中我解析了我的日志，我还分离了我的日志，将每个日志作为自己的记录而不是嵌套数组看到： How to ship logs from pods on Kubernetes running on top of GCP to elasticsearch/logstash?

我在logstash中的配置用于解析日志：

filter {
  if [resource][type] == "gae_app" {
    # split the protoPayload.line array, so each log message is a separate entry in Elasticsearch
    split {
      field => "[protoPayload][line]"
      target => "line"
      remove_field => [  "httpRequest", "operation", "protoPayload"]
    }
    # extract `line.logMessage` and `line.severity` fields
    mutate {
      add_field => {"logMessage" => "%{[line][logMessage]}"}
      replace => {"severity" => "%{[line][severity]}"}
      remove_field => ["line"]
    }
    # remove the `com.example.MyClass myFunc: ` prefix from log
    grok {
      match => { "logMessage" => "^%{DATA}: %{GREEDYDATA:parsedMessage}"}
    }
    # parse the log message into json, json fields will be located in root
    json {
      source => "parsedMessage"
      target => "jsonPayload"
      add_field => {"[jsonPayload][level]" => "%{severity}"}
      remove_field => ["parsedMessage", "logMessage"]
    }

    # uniform GAE logs to the structure of GKE logs
    grok {
      match => {  # check..
        "[resource][labels][version_id]" =>
        "^%{DATA:[resource][labels][container_name]}-%{GREEDYDATA:[resource][labels][namespace_id]}"}
    }


  }
}

将appengine java上的日志写入stackdriver-logging

1 个答案: