Question

为了记录在ASP.NET Core Web应用程序中花费在SQL Server查询上的时间，我一直在使用以下代码，已在某些中间件中订阅了所有DiagnosticListeners并使用了{{1} }。

我不确定这是否是性能方面的最佳解决方案，并且想知道是否有通过直接从EFCore捕获详细的日志记录对象来使用ASP.NET Core Logging API的更好方法？理想情况下，我希望保留通过一个请求执行的所有查询的总时间，并在请求结束时保留总计（以毫秒为单位），中间件可以使用该时间。

Observable

Answer 1

衡量性能的方法有很多，但是在计时查询方面，我从SQL Profiler之类的数据库服务器工具开始。对于SQL Server，SQL Server Profiler和SQL Server Management Studio工具提供了无穷无尽的细节。

现在，我无法对您的代码进行特别评论，但也许可以给您一些值得考虑的建议。如果我想用代码记录查询的执行时间，那么通常我会做两件事之一。

1）经过测试的秒表永远不会失败。就像你在做。

2）现在，我最喜欢的工具。如果可以，我在项目中使用MiniProfiler。它来自Stack Exchange，后者是运行此网站的人员，我们现在正在讨论配置文件。它向您显示SQL查询并警告您常见的错误。您可以将其与EF Core结合使用，并且它具有漂亮的小部件，您可以轻松地将其集成到网站上的页面中以随时查看正在发生的事情。

Answer 2

分享我在项目中所做的事情。

为每个传入请求创建ApiRequest对象。
从请求中收集基本信息，例如IP地址，用户代理，用户ID，引荐来源网址，UriPath，搜索类型等。
现在，我可以在控制器/服务层中引用ApiRequest对象，并且启动/停止计时器并计算每个步骤的时间（分贝通话）
将ApiRequest对象保存到首选日志记录数据库在做出回应之前。

ApiRequest类：

public class ApiRequest : IDisposable
{
    [BsonId]
    public string Id { get; set; }
    // public string RequestId { get; set; }
    public string ClientSessionId { get; set; }
    public DateTime RequestStartTime { get; set; }
    public string LogLevel { get; set; }
    public string AccessProfile { get; set; }
    public string ApiClientIpAddress { get; set; }
    public GeoData ApiClientGeoData { get; set; }
    public string OriginatingIpAddress { get; set; }
    public GeoData OriginatingGeoData { get; set; }
    public int StatusCode { get; set; }
    public bool IsError { get; set; }
    // public string ErrorMessage { get; set; }
    public ConcurrentBag<string> Errors { get; set; }
    public ConcurrentBag<string> Warnings { get; set; }
    public long TotalExecutionTimeInMs { get; set; }
    public long? ResponseSizeInBytes { get; set; }
    public long TotalMySqlSpCalls { get; set; }
    public long DbConnectionTimeInMs { get; set; }
    public long DbCallTimeInMs { get; set; }
    public long OverheadTimeInMs { get; set; }
    public string UriHost { get; set; }
    public string UriPath { get; set; }
    public string SearchRequest { get; set; }
    public string Referrer { get; set; }
    public string UserAgent { get; set; }
    public string SearchType { get; set; }
    public ConcurrentQueue<RequestExecutionStatistics> ExecutionHistory { get; set; }
    public DateTime CreatedDate { get; set; }
    public string CreatedBy { get; set; }

    [BsonIgnore]
    public Stopwatch Timer { get; set; }

    public ApiRequest(Stopwatch stopwatch)
    {
        Id = Guid.NewGuid().ToString();
        // RequestId = Guid.NewGuid().ToString();
        Timer = stopwatch;
        RequestStartTime = DateTime.Now.Subtract(Timer.Elapsed);
        ExecutionHistory = new ConcurrentQueue<RequestExecutionStatistics>();
        ExecutionHistory.Enqueue(new RequestExecutionStatistics
        {
            Description = "HTTP Request Start",
            Status = RequestExecutionStatus.Started,
            Index = 1,
            StartTime = Timer.ElapsedMilliseconds,
            ExecutionTimeInMs = 0
        });
        Errors = new ConcurrentBag<string>();
        Warnings = new ConcurrentBag<string>();
    }

    public Task AddExecutionTimeStats(string description, RequestExecutionStatus status, long startTime, long executionTimeInMs)
    {
        int count = ExecutionHistory.Count;

        return Task.Run(() =>
            ExecutionHistory.Enqueue(new RequestExecutionStatistics
            {
                Description = description,
                Status = status,
                Index = count + 1,
                StartTime = startTime,
                ExecutionTimeInMs = executionTimeInMs
            }));
    }

    public Task UpdateExecutionTimeStats(string description, long executionTimeInMs)
    {
        return Task.Run(() =>
        {
            var stats = ExecutionHistory.FirstOrDefault(e => e.Description.ToLower() == description.ToLower());
            if (stats != null) stats.ExecutionTimeInMs = executionTimeInMs;
        });
    }

    #region IDisposable implementation

    private bool _disposed;

    // Public implementation of Dispose pattern callable by consumers.
    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    // Protected implementation of Dispose pattern.
    protected virtual void Dispose(bool disposing)
    {
        if (_disposed) { return; }

        if (disposing)
        {
            // Free managed objects here.
            Timer.Stop(); // Probably doesn't matter, but anyways...
        }

        // Free any unmanaged objects here.


        // Mark this object as disposed.
        _disposed = true;
    }

    #endregion
}

样品控制器：

[Route("api/[controller]")]
[Authorize(Policy = Constants.Policies.CanAccessSampleSearch)]
public class SampleSearchController : Controller
{
    private readonly AppSettings _appSettings;
    private readonly ISampleSearchService _sampleService;

    public SampleSearchController(IOptions<AppSettings> appSettings, ISampleSearchService sampleSearchService)
    {
        _appSettings = appSettings.Value;
        _sampleService = sampleSearchService;
    }

    [HttpPost]
    public async Task<Response> PostAsync([FromBody]PersonRequest request)
    {
        var apiRequest = HttpContext.Items[Constants.CacheKeys.SampleApiRequest] as ApiRequest;
        await apiRequest.AddExecutionTimeStats("Sample Search Controller", RequestExecutionStatus.Started,
            apiRequest.Timer.ElapsedMilliseconds, 0);

        var result = new Response();

        if (!ModelState.IsValid)
        {
            apiRequest.StatusCode = (int)HttpStatusCode.BadRequest;
            apiRequest.IsError = true;

            foreach (var modelState in ModelState.Values)
            {
                foreach (var error in modelState.Errors)
                {
                    apiRequest.Errors.Add(error.ErrorMessage);
                }
            }
        }
        else
        {
            result = await _sampleService.Search(request);
        }

        await apiRequest.AddExecutionTimeStats("Sample Search Controller", RequestExecutionStatus.Complted, 0,
            apiRequest.Timer.ElapsedMilliseconds);

        return result;
    }
}

样品服务：

public class SampleSearchService : ISampleSearchService
{
    #region Variables

    private readonly AppSettings _appSettings;
    private readonly IStateService _stateService;
    private readonly IDistrictService _districtService;
    private readonly IHttpContextAccessor _httpContextAccessor;

    #endregion Variables

    public SampleSearchService(IOptions<AppSettings> appSettings, IStateService stateService,
        IDistrictService districtService, IHttpContextAccessor httpContextAccessor)
    {
        _appSettings = appSettings.Value;
        _stateService = stateService;
        _districtService = districtService;
        _httpContextAccessor = httpContextAccessor;
    }

    public async Task<Response> Search(PersonRequest request)
    {
        var apiRequest =
            _httpContextAccessor.HttpContext.Items[Constants.CacheKeys.SampleApiRequest] as ApiRequest;

        await apiRequest.AddExecutionTimeStats("Sample Search Service", RequestExecutionStatus.Started,
            apiRequest.Timer.ElapsedMilliseconds, 0);

        var response = new Response();

        using (var db = new ApplicationDb(_appSettings.ConnectionStrings.MyContext))
        {
            var dbConnectionStartTime = apiRequest.Timer.ElapsedMilliseconds;
            await db.Connection.OpenAsync();
            apiRequest.DbConnectionTimeInMs = apiRequest.Timer.ElapsedMilliseconds - dbConnectionStartTime;

            await apiRequest.AddExecutionTimeStats("DB Connection", RequestExecutionStatus.Complted,
                dbConnectionStartTime, apiRequest.DbConnectionTimeInMs);

            using (var command = db.Command(_appSettings.StoredProcedures.GetSampleByCriteria,
                _appSettings.Timeouts.GetSampleByCriteriaTimeout, CommandType.StoredProcedure, db.Connection))
            {
                command.Parameters.Add(new MySqlParameter
                {
                    ParameterName = "p_Id",
                    DbType = DbType.Int64,
                    Value = request.Id
                });                    

                try
                {
                    await apiRequest.AddExecutionTimeStats("Sample DB Call", RequestExecutionStatus.Started,
                        apiRequest.Timer.ElapsedMilliseconds, 0);

                    response = await ReadAllAsync(await command.ExecuteReaderAsync());

                    await apiRequest.AddExecutionTimeStats("Sample DB Call", RequestExecutionStatus.Complted, 0,
                        apiRequest.Timer.ElapsedMilliseconds);
                }
                catch (Exception e)
                {
                    apiRequest.StatusCode = (int)HttpStatusCode.InternalServerError;
                    apiRequest.IsError = true;
                    apiRequest.Errors.Add(e.Message);
                }
            }
        }

        apiRequest.SearchRequest = JsonConvert.SerializeObject(request);
        await apiRequest.AddExecutionTimeStats("Sample Search Service", RequestExecutionStatus.Complted, 0,
            apiRequest.Timer.ElapsedMilliseconds);
        return response;
    }

    private async Task<Response> ReadAllAsync(DbDataReader reader)
    {
        var SampleResponse = new Response();

        using (reader)
        {
            while (await reader.ReadAsync())
            {
                SampleResponse.AvailableCount = Convert.ToInt32(await reader.GetFieldValueAsync<long>(0));

                if (reader.NextResult())
                {
                    while (await reader.ReadAsync())
                    {
                        var SampleRecord = new Record()
                        {                                
                            District = !await reader.IsDBNullAsync(1) ? await reader.GetFieldValueAsync<string>(6) : null,
                            State = !await reader.IsDBNullAsync(2) ? await reader.GetFieldValueAsync<string>(7) : null
                        };

                        SampleResponse.Records.Add(SampleRecord);
                    }
                }
            }
        }

        return SampleResponse;
    }
}

ApiRequestsMiddleware：

public class ApiRequestsMiddleware
{
    private readonly RequestDelegate _next;
    private readonly IApiRequestService _apiRequestService;

    public ApiRequestsMiddleware(RequestDelegate next, IApiRequestService apiRequestService)
    {
        _next = next;
        _apiRequestService = apiRequestService;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var stopwatch = Stopwatch.StartNew();
        var accessProfileName = context.User.Claims != null && context.User.Claims.Any()
            ? context.User.Claims.FirstOrDefault(c => c.Type == ClaimTypes.Name)?.Value
            : null;

        try
        {
            var request = new ApiRequest(stopwatch)
            {
                AccessProfile = accessProfileName,
                ClientSessionId = ContextHelper.GetHeadersValue(context, Constants.Headers.ClientSessionId),
                ApiClientIpAddress = context.Connection.RemoteIpAddress.ToString()
            };

            var originatingIpAddress = ContextHelper.GetHeadersValue(context, Constants.Headers.OriginatingIpAddress);
            request.OriginatingIpAddress = !string.IsNullOrWhiteSpace(originatingIpAddress)
                ? originatingIpAddress
                : context.Connection.RemoteIpAddress.ToString();

            request.UriHost = context.Request.Host.ToString();
            request.UriPath = context.Request.Path;

            var referrer = ContextHelper.GetHeadersValue(context, Constants.Headers.Referrer);
            request.Referrer = !string.IsNullOrWhiteSpace(referrer) ? referrer : null;

            var userAgent = ContextHelper.GetHeadersValue(context, Constants.Headers.UserAgent);
            request.UserAgent = !string.IsNullOrWhiteSpace(userAgent) ? userAgent : null;

            request.SearchType = SearchHelper.GetSearchType(request.UriPath).ToString();

            context.Items.Add(Constants.CacheKeys.SampleApiRequest, request);

            await _next(context);
            stopwatch.Stop();

            request.StatusCode = context.Response.StatusCode;
            request.LogLevel = LogLevel.Information.ToString();
            request.TotalExecutionTimeInMs = stopwatch.ElapsedMilliseconds;

            if (request.IsError)
                request.LogLevel = LogLevel.Error.ToString();

            if (_apiRequestService != null)
                Task.Run(() => _apiRequestService.Create(request));
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
            throw;
        }
    }
}

public static class ApiRequestsMiddlewareExtensions
{
    public static IApplicationBuilder UseApiRequests(this IApplicationBuilder builder)
    {
        return builder.UseMiddleware<ApiRequestsMiddleware>();
    }
}

ApiRequestService（用于将日志条目插入db）：

public class ApiRequestService : IApiRequestService
{
    private readonly IMongoDbContext _context;
    private readonly IIpLocatorService _ipLocatorService;

    public ApiRequestService(IMongoDbContext context, IIpLocatorService ipLocatorService)
    {
        _context = context;
        _ipLocatorService = ipLocatorService;
    }

    public async Task<IEnumerable<ApiRequest>> Get()
    {
        return await _context.ApiRequests.Find(_ => true).ToListAsync();
    }

    public async Task<ApiRequest> Get(string id)
    {
        var filter = Builders<ApiRequest>.Filter.Eq("Id", id);

        try
        {
            return await _context.ApiRequests.Find(filter).FirstOrDefaultAsync();
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
            throw;
        }
    }

    public async Task Create(ApiRequest request)
    {
        try
        {
            request.OriginatingGeoData = null; // await _ipLocatorService.Get(request.OriginatingIpAddress);

            var finalHistory = new ConcurrentQueue<RequestExecutionStatistics>();

            foreach (var history in request.ExecutionHistory.ToList())
            {
                if (history.Status == RequestExecutionStatus.Started)
                {
                    var findPartner = request.ExecutionHistory.FirstOrDefault(e =>
                        e.Description.ToLower() == history.Description.ToLower() &&
                        e.Status == RequestExecutionStatus.Complted);

                    if (findPartner != null)
                    {
                        var temp = history.Clone();
                        temp.Status = RequestExecutionStatus.Complted;
                        temp.ExecutionTimeInMs = findPartner.ExecutionTimeInMs - history.StartTime;
                        finalHistory.Enqueue(temp);
                    }
                    else
                        finalHistory.Enqueue(history);
                }
                else if (history.Status == RequestExecutionStatus.Complted)
                {
                    var findPartner = request.ExecutionHistory.FirstOrDefault(e =>
                        e.Description.ToLower() == history.Description.ToLower() &&
                        e.Status == RequestExecutionStatus.Started);

                    if (findPartner == null)
                        finalHistory.Enqueue(history);
                }
            }

            request.ExecutionHistory = finalHistory;
            request.CreatedDate = DateTime.Now;
            request.CreatedBy = Environment.MachineName;
            await _context.ApiRequests.InsertOneAsync(request);
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
            throw;
        }
    }
}

在Configure方法中注册：

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
    {
        // Other code        

        app.UseApiRequests();
        app.UseResponseWrapper();
        app.UseMvc();
    }

示例日志输出：

{
"_id" : "75a90cc9-5d80-4eb8-aae1-70a3611fe8ff",
"RequestId" : "98be1d85-9941-4a17-a73a-5f0a38bd7703",
"ClientSessionId" : null,
"RequestStartTime" : ISODate("2019-09-11T15:22:05.802-07:00"),
"LogLevel" : "Information",
"AccessProfile" : "Sample",
"ApiClientIpAddress" : "0.0.0.0",
"ApiClientGeoData" : null,
"OriginatingIpAddress" : "0.0.0.0",
"OriginatingGeoData" : null,
"StatusCode" : 200,
"IsError" : false,
"Errors" : [],
"Warnings" : [],
"TotalExecutionTimeInMs" : NumberLong(115),
"ResponseSizeInBytes" : NumberLong(0),
"TotalMySqlSpCalss" : NumberLong(0),
"DbConnectionTimeInMs" : NumberLong(3),
"DbCallTimeInMs" : NumberLong(0),
"OverheadTimeInMs" : NumberLong(0),
"UriHost" : "www.sampleapi.com",
"UriPath" : "/api/Samples",
"SearchRequest" : null,
"Referrer" : null,
"UserAgent" : null,
"SearchType" : "Sample",
"ExecutionHistory" : [ 
    {
        "Description" : "HTTP Request Start",
        "Index" : 1,
        "StartTime" : NumberLong(0),
        "ExecutionTimeInMs" : NumberLong(0)
    }, 
    {
        "Description" : "Sample Search Controller",
        "Index" : 2,
        "StartTime" : NumberLong(0),
        "ExecutionTimeInMs" : NumberLong(115)
    }, 
    {
        "Description" : "Sample Search Service",
        "Index" : 3,
        "StartTime" : NumberLong(0),
        "ExecutionTimeInMs" : NumberLong(115)
    }, 
    {
        "Description" : "DB Connection",
        "Index" : 4,
        "StartTime" : NumberLong(0),
        "ExecutionTimeInMs" : NumberLong(3)
    }, 
    {
        "Description" : "Sample DB Call",
        "Index" : 5,
        "StartTime" : NumberLong(3),
        "ExecutionTimeInMs" : NumberLong(112)
    }
],
"CreatedDate" : ISODate("2019-09-11T15:22:05.918-07:00"),
"CreatedBy" : "Server"}

Answer 3

public class QueryInterceptor
    {
        private readonly ILogger<QueryInterceptor> _logger;
        private string _query;
        private DateTimeOffset _startTime;
        public QueryInterceptor(ILogger<QueryInterceptor> logger)
        {
            _logger = logger;
        }

        [DiagnosticName("Microsoft.EntityFrameworkCore.Database.Command.CommandExecuting")]
        public void OnCommandExecuting(DbCommand command, DbCommandMethod executeMethod, Guid commandId, Guid connectionId, bool async, DateTimeOffset startTime)
        {
            _query = command.CommandText;
            _startTime = startTime;
        }

        [DiagnosticName("Microsoft.EntityFrameworkCore.Database.Command.CommandExecuted")]
        public void OnCommandExecuted(object result, bool async)
        {
            var endTime = DateTimeOffset.Now;
            var queryTiming = (endTime - _startTime).TotalSeconds;
            _logger.LogInformation("\n" + "Executes " + "\n" + _query + "\n" + "in " + queryTiming + " seconds\n");
        }

        [DiagnosticName("Microsoft.EntityFrameworkCore.Database.Command.CommandError")]
        public void OnCommandError(Exception exception, bool async)
        {
            _logger.LogError(exception.Message);
        }
    }

和Program.cs

var loggerQueryInterceptor = services.GetService<ILogger<QueryInterceptor>>();
var listener = context.GetService<DiagnosticSource>();
(listener as DiagnosticListener).SubscribeWithAdapter(new QueryInterceptor(loggerQueryInterceptor));

Answer 4

查询性能应在两个级别上完成，对控制器/服务操作进行计时，以及在数据库级别进行性能分析。性能问题的产生有多种原因，并且可能仅在控制器或数据库中或同时在两者中被检测到。在代码级别记录性能指标应该始终可配置，以便能够轻松关闭和打开它，因为任何性能捕获都代表性能下降，再加上记录结果所需的资源空间。

冒着有点偏离主题的风险，我概述了我遇到的典型性能陷阱以及如何测量/检测它们。目的只是概述为什么SQL端概要分析在基于端的基于代码的计时器中很有价值，以检测潜在的性能问题，并确定解决这些问题的步骤。

流氓惰性加载调用。通过SQL分析进行检测。 Web请求在初始查询快速完成的地方查询数据，Web请求在调试器中似乎快速结束，但是客户端在一段时间内未收到响应。在初始查询和请求完成之后，事件探查器跟踪许多“按ID”查询。这些是由序列化程序触发的惰性加载调用。原因：将实体传递给视图/消费者的反模式。解决方案：使用投影来填充视图模型。
昂贵的查询。通过代码计时器和SQL分析进行检测。这些查询需要大量时间才能运行，这可以通过调试来检测。这些查询还在分析器结果中显示可检测的执行时间。解决方案的范围从检查执行的查询以改进执行计划（缺少索引/改进的索引）到确定查询是否可以简化或在代码中分解。
间歇性慢速查询。通过代码计时器和SQL分析进行检测。这些可以通过检查分析结果提前获取。讲故事的标志是涉及大量行触摸计数的查询。在低负载情况下运行时，这些查询可以具有非常合理的执行时间。在较高的加载时间中，执行时间会急剧增加。涉及很多行的查询冒着用行锁相互绑在一起的风险，这些行锁需要避免脏读。锁定会导致延迟和潜在的死锁。解决方案包括：优化查询以仅拉回它们以更好地利用索引所需的数据；对于用于报表和搜索的扩展查询，请考虑在大型系统中命中只读副本。
EF“主义”。由代码计时器检测，而不是性能分析。投影实体图时，某些EF查询可能会变得有些奇怪。基础查询运行速度很快，并且不会产生其他查询，但是EF查询在返回之前会停滞不前。有时，这些可能是由于复杂的Linq表达式导致的查询看起来很合理，但它似乎就在那里。有时，原因可能非常明显，例如长期存在的DbContext最终跟踪了整个数据库的大部分。其他时候，需要逐案分析解决方案。它们通常涉及使用/更改投影以简化从EF返回的数据，或将查询分解为更简单的部分，即使它导致两个查询而不是一个查询。

使用EF Core记录查询持续时间

4 个答案: