我编写了c#WebAPI,并希望使用log4net将请求记录到文件中。将日志写入日志文件后,我想将这些日志文件(将使用DelegateHandler捕获所有请求)推送到ElasticSearch。我到处搜索,但我无法找到解决问题的方法。以下是我拥有的log4net.config文件(ElasticAppender不起作用)
<log4net>
<root>
<appender-ref ref="FileAppender" />
<appender-ref ref="ElasticSearchAppender" />
</root>
<appender name="FileAppender" type="log4net.Appender.RollingFileAppender">
<file type="log4net.Util.PatternString" value="Logs/API_.log" />
<appendToFile value="true" />
<rollingStyle value="Composite" />
<datePattern value="yyyy-MM-dd" />
<maximumFileSize value="10MB" />
<preserveLogFileNameExtension value="true"/>
<staticLogFileName value="false" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date{HH:mm:ss.fff}, %message%newline" />
</layout>
<filter type="log4net.Filter.LevelRangeFilter">
<levelMin value="DEBUG" />
<levelMax value="ERROR" />
</filter>
</appender>
<appender name="ElasticSearchAppender" type="log4net.ElasticSearch.ElasticSearchAppender, log4stash">
<Server>elasticnode01</Server>
<Port>9200</Port>
<IndexName>apilogentry</IndexName>
<ElasticFilters>
<Filter type="log4net.ElasticSearch.Filters.RenameKeyFilter, log4stash">
<Key>processId</Key>
<RenameTo>ServiceName</RenameTo>
</Filter>
<Filter type="log4net.ElasticSearch.Filters.RenameKeyFilter, log4stash">
<Key>Message</Key>
<RenameTo>message</RenameTo>
</Filter>
<Grok>
<SourceKey>message</SourceKey>
<Pattern>%{NUMBER:ResponseCode} %{WORD:Method} %{URIPATHPARAM:Url} %{NUMBER:ElapsedMls} %{GREEDYDATA:StatusMessage}</Pattern>
</Grok>
</ElasticFilters>
<filter type="log4net.Filter.LevelRangeFilter">
<levelMin value="INFO" />
<levelMax value="ERROR" />
</filter>
</appender>
</log4net>
这是我用来收集数据的类(我想记录)。
public class ApiLogEntry
{
public long ApiLogEntryId { get; set; } // The (database) ID for the API log entry.
public string Application { get; set; } // The application that made the request.
public string User { get; set; } // The user that made the request.
public string Machine { get; set; } // The machine that made the request.
public string RequestIpAddress { get; set; } // The IP address that made the request.
public string RequestContentType { get; set; } // The request content type.
public string RequestContentBody { get; set; } // The request content body.
public string RequestUri { get; set; } // The request URI.
public string RequestMethod { get; set; } // The request method (GET, POST, etc).
public string RequestRouteTemplate { get; set; } // The request route template.
public string RequestRouteData { get; set; } // The request route data.
public string RequestHeaders { get; set; } // The request headers.
public DateTime? RequestTimestamp { get; set; } // The request timestamp.
public string ResponseContentType { get; set; } // The response content type.
public string ResponseContentBody { get; set; } // The response content body.
public int? ResponseStatusCode { get; set; } // The response status code.
public string ResponseHeaders { get; set; } // The response headers.
public DateTime? ResponseTimestamp { get; set; } // The response timestamp.
}
这是处理所有HTTP请求的类,应该记录信息。现在,信息正在记录到文件中。
public class ApiLogHandler : DelegatingHandler
{
ILogger _logger;
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var apiLogEntry = CreateApiLogEntryWithRequestData(request);
if (request.Content != null)
{
await request.Content.ReadAsStringAsync()
.ContinueWith(task =>
{
apiLogEntry.RequestContentBody = task.Result;
}, cancellationToken);
}
return await base.SendAsync(request, cancellationToken)
.ContinueWith(task =>
{
var response = task.Result;
// Update the API log entry with response info
apiLogEntry.ResponseStatusCode = (int)response.StatusCode;
apiLogEntry.ResponseTimestamp = DateTime.Now;
if (response.Content != null)
{
// apiLogEntry.ResponseContentBody = response.Content.ReadAsStringAsync().Result;
apiLogEntry.ResponseContentType = response.Content.Headers.ContentType.MediaType;
apiLogEntry.ResponseHeaders = SerializeHeaders(response.Content.Headers);
}
// TODO: Save the API log entry to the database
_logger.Info(apiLogEntry);
return response;
}, cancellationToken);
}
private ApiLogEntry CreateApiLogEntryWithRequestData(HttpRequestMessage request)
{
var context = ((HttpContextBase)request.Properties["MS_HttpContext"]);
var routeData = request.GetRouteData();
return new ApiLogEntry
{
Application = "Search.API",
User = context.User.Identity.Name,
Machine = Environment.MachineName,
RequestContentType = context.Request.ContentType,
RequestRouteTemplate = routeData.Route.RouteTemplate,
//RequestRouteData = SerializeRouteData(routeData),
RequestIpAddress = context.Request.UserHostAddress,
RequestMethod = request.Method.Method,
RequestHeaders = SerializeHeaders(request.Headers),
RequestTimestamp = DateTime.Now,
RequestUri = request.RequestUri.ToString()
};
}
private string SerializeRouteData(IHttpRouteData routeData)
{
return JsonConvert.SerializeObject(routeData, Formatting.None, new JsonSerializerSettings()
{
ReferenceLoopHandling = ReferenceLoopHandling.Ignore
});
//return JsonConvert.SerializeObject(routeData, Formatting.Indented, new JsonSerializerSettings
//{
// PreserveReferencesHandling = PreserveReferencesHandling.Objects
//});
}
private string SerializeHeaders(HttpHeaders headers)
{
var dict = new Dictionary<string, string>();
foreach (var item in headers.ToList())
{
if (item.Value != null)
{
var header = String.Empty;
foreach (var value in item.Value)
{
header += value + " ";
}
// Trim the trailing space and add item to the dictionary
header = header.TrimEnd(" ".ToCharArray());
dict.Add(item.Key, header);
}
}
return JsonConvert.SerializeObject(dict, Formatting.Indented);
}
public ApiLogHandler(ILogger logger)
{
_logger = logger;
}
}
如何从这些文件中获取输出到ElasticSearch?我已经阅读了有关FileBeat和logstash的内容,但不知道如何让这一切全部运行。任何人对我可能缺少的东西都有任何想法? (我知道log4net.config中的grok条目是对的(我试图映射到ApiLogEntry类但不知道如何继续)。
TIA
答案 0 :(得分:0)
遵循以下文档 Log4net.ElasticSearch NuGet Lib,您应该能够实现自己的目标。
答案 1 :(得分:0)
Elastic Search Log4Net附加程序does not recommend using this approach in production的文档。
建议使用Filebeat将日志文件数据通过LogStash(用于数据转换/解析)推入Elastic Search数据库。在撰写本文时,所有这些组件都可以在self hosted Basic installation上自由使用。