logstash @timestamp的弹性搜索映射到.NET中的强类型

时间:2016-08-17 19:14:17

标签: c# elasticsearch nest serilog elasticsearch-net

我使用Serilog和ElasticSearch Sink与AutoRegisterTemplate = true

将以下内容存储在ElasticSearch



{
"@timestamp": "2016-08-17T08:57:37.3487446+02:00",
"level": "Information",
"messageTemplate": "User login {UserId}",
"message": "User login ...,
"fields": {
  "UserId": "...",
  "ProcessId": 15568,
  "ThreadId": 14,
  "MachineName": "...",
  "EnvironmentUserName": "...",
  "HttpRequestId": "...",
  "HttpRequestClientHostIP": "::1",
  "HttpRequestType": "POST",
  "Version": "1.0.0.0"
  }
}




我可以使用DynamicResponse

查询索引
            var searchResponse = client.Search<DynamicResponse>(s => s
            .Index("logstash-*")
            .AllTypes()
            .Size(50)
            .Query(q => q.Bool(b => b.Must(bs => bs.Term(p => p.Field("fields.UserId.raw").Value("..."))))
            )
        );

但我想为结果使用强类型类。

我尝试过以下课程。

        public class LogResponse
    {
        public string Message { get; set; }
        [Date(Name = "@timestamp")]
        public DateTime Timestamp { get; set; }
        public LogResponseFields Fields { get; set; }

        public class LogResponseFields
        {
            public Guid UserId { get; set; }
        }
    }

但是没有设置时间戳,它默认为DateTime.Min,Message和UserId都被正确映射。

如何映射@timestamp字段?

3 个答案:

答案 0 :(得分:0)

尝试DateTimeOffset,它似乎是带有嵌入式时区信息的时间戳。 (具体为+02:00

答案 1 :(得分:0)

时间戳字段的映射正常工作;这里有一个完整的例子来演示

subset(as.data.frame(table(df)), Freq !=0)

此输出

void Main()
{
    var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
    var defaultIndex = "logstash-" + DateTime.UtcNow.ToString("yyyy-MM-dd");
    var connectionSettings = new ConnectionSettings(pool)
            .DefaultIndex(defaultIndex)
            .InferMappingFor<LogResponse>(m => m
                .TypeName("log")
            );

    var client = new ElasticClient(connectionSettings);

    // delete the index if it exists. Useful for demo purposes
    if (client.IndexExists(defaultIndex).Exists)
    {
        client.DeleteIndex(defaultIndex);
    }

    // use the lowlevel client to send the exact json as 
    // it would be sent from the source
    var createResponse = client.LowLevel.Index<string>(
        defaultIndex, 
        "log",
        @"{
            ""@timestamp"": ""2016-08-17T08:57:37.3487446+02:00"",
            ""level"": ""Information"",
            ""messageTemplate"": ""User login {UserId}"",
            ""message"": ""User login .."",
            ""fields"": {
              ""UserId"": ""29a35806-15d2-4eed-a3bf-707760e426b8"",
              ""ProcessId"": 15568,
              ""ThreadId"": 14,
              ""MachineName"": ""..."",
              ""EnvironmentUserName"": ""..."",
              ""HttpRequestId"": ""..."",
              ""HttpRequestClientHostIP"": ""::1"",
              ""HttpRequestType"": ""POST"",
              ""Version"": ""1.0.0.0""
            }
        }");

    if (!createResponse.SuccessOrKnownError)
    {
        Console.WriteLine("Error indexing document");
    }

    // Refresh the index after indexing. Useful for demo purposes.
    client.Refresh(defaultIndex);

    var searchResponse = client.Search<LogResponse>(s => s
        .AllTypes()
        .Size(50)
        .Query(q => q
            .MatchAll()
        )
    );

    foreach (var document in searchResponse.Documents)
    {
        Console.WriteLine(document.Timestamp);
    }
}

public class LogResponse
{
    public string Message { get; set; }
    [Date(Name = "@timestamp")]
    public DateTime Timestamp { get; set; }
    public LogResponseFields Fields { get; set; }

    public class LogResponseFields
    {
        public Guid UserId { get; set; }
    }
}

对我来说(它已被反序列化到我的本地17/08/2016 4:57:37 PM )。

答案 2 :(得分:0)

感谢您的投入。

当我将示例发布到SO时,问题得到解决。

我的真实代码是:

public class LogResponse : DynamicResponse

通过删除DynamicResponse inheritence

public class LogResponse

每件事都有效。