Hangfire后台作业抛出“内存不足,无法继续执行程序。”

时间:2019-03-08 15:36:18

标签: c# multithreading asp.net-core hangfire parallel.foreach

我很难弄清楚这一点,也许有人可以在这里帮助我。

我有一个Asp.net Core 2.0 Web应用程序,我正在使用Hangfire在后台处理作业,通常是在不同平台之间同步产品或订单,但是有一项作业始终会因“内存不足而失败”。继续执行程序。”还有“对象引用未设置为对象的实例”,我将跟踪记录下来,并将我带到引发错误的Parallel.ForEach上,现在当我手动执行相同操作(即不执行hangfire)时,我不会例外,一切正常。

所以让我在这里分享一些代码。

所以首先我们像这样设置hangfire。在启动类中,通过配置方法,我们执行此操作。

            app.UseHangfireDashboard("/tasks", new DashboardOptions() {
            Authorization = new[] { new HangFireAuthorizationFilter() }
        });

            var schedules = new ScheduleTasks(app.ApplicationServices);
            schedules.RegisterRecurentJobs();

现在ScheduleTasks看起来像这样。

    public class ScheduleTasks
{
    private readonly IApiService _apiService;
    private readonly ISettingService _settingService;

    public ScheduleTasks(IServiceProvider serviceProvider)
    {
        var scope = serviceProvider.CreateScope();
        _apiService = scope.ServiceProvider.GetService<IApiService>();
        _settingService = scope.ServiceProvider.GetService<ISettingService>();
    }

    public void RegisterRecurentJobs()
    {
        var settings = _settingService.LoadSetting<ApiSettings>();

        if(!settings.SilentMode)
        {
            RecurringJob.AddOrUpdate<IApiService>("Config", x => x.SyncConfig(), "* 1 * * *");
            RecurringJob.AddOrUpdate<IApiService>("Manufacturers", x => x.SyncManufacturers(), "* 2 * * *");
            RecurringJob.AddOrUpdate<IApiService>("Products", x => x.SyncProducts(30, false), "* 3 * * *");
        }

    }
}

现在这是我执行SyncProducts的api服务方法。

        public async Task SyncProducts(int days, bool hasSalesOnly = true)
    {

        var s = new StringBuilder();
        try
        {
            var search = new SearchModel()
            {
                ModifiedDate = new DateRange()
                {
                    FromDate = DateTime.Today.AddDays(-days)
                },
                PageSize = 300000,
                HaveSalesOnly = hasSalesOnly
            };

            var ids = await _productService.GetProductFeed(search);
            var hasNext = ids?.Result?.AdditionalResultsId;
            var failedIds = new List<string>();
            var retry = true;

            while (ids.Result.Ids != null && ids.Result.Ids.Count > 0)
            {
                var parallelOptions = new ParallelOptions { MaxDegreeOfParallelism = 20 };

                s.Append("[");
                var start = true;

                Parallel.ForEach(ids.Result.Ids, parallelOptions, id =>
                {
                    var itemResult = _productService.GetProductByProductId(id).Result;
                    if (itemResult.Success)
                    {
                        lock (s)
                        {
                            if (start)
                            {
                                s.Append(itemResult.ResponseString);
                            }
                            else
                            {
                                var r = "," + itemResult.ResponseString;
                                s.Append(r);
                            }

                            start = false;
                        }
                    }
                    else
                    {
                        failedIds.Add(id);
                    }

                    itemResult = null;

                });

                s.Append("]");

                var newFileName = "result.json";
                var newFileDir = GetJsonFileDirectoryPath() + newFileName;
                var baseDir = Environment.CurrentDirectory + "/wwwroot";
                var dataPath = baseDir + newFileDir;
                File.WriteAllText(dataPath, s.ToString());
                s = null;

                ids.Result.Ids = null;

                var param = new SqlParameter
                {
                    ParameterName = "path",
                    SqlDbType = SqlDbType.VarChar,
                    Direction = ParameterDirection.Input,
                    Value = dataPath
                };

                var imagesToProcess = await _itemRepository.ExecuteStoredProcedureList<SyncItems>("[dbo].[SyncItems]", param);

                foreach (var image in imagesToProcess)
                {
                    if (image.ProductId == null)
                        continue;

                    //for now delete all images, and save the new ones.
                    var imagesToDelete = _pictureService.GetProductPicturesByProductId(image.ProductId.Value);

                    foreach (var i in imagesToDelete)
                    {
                        await _pictureService.DeleteProductPicture(i);
                    }

                    if (image.ImageUrls != null)
                    {
                        var imageList = image.ImageUrls.Split(',');

                        foreach (var i in imageList)
                        {
                            await _pictureService.InsertProductPictureFromPictureUrl(i, image.ProductId.Value);
                        }
                    }


                }

                if (!string.IsNullOrEmpty(hasNext))
                {
                    ids = await _productService.GetNextPage(ids.Result.AdditionalResultsId);
                    hasNext = ids.Result.AdditionalResultsId;
                }
                else
                {
                    ids.Result.Ids = null;
                }

                if (string.IsNullOrEmpty(hasNext) && ids.Result.Ids == null && retry ||
                    string.IsNullOrEmpty(hasNext) && ids.Result.Ids.Count == 0 && retry)
                {
                    ids = new BaseSearchResult();
                    ids.Result.Ids = failedIds;

                    failedIds = new List<string>();
                    retry = false;
                }
            }

            ids = null;

        }
        catch (Exception ex)
        {
            s = null;
            aEvent = new ApiEvent()
            {
                Level = LogLevel.Critical,
                EventDate = DateTime.Now,
                User = "Api",
                EventType = EventTypeEnum.Product,
                EntityTypeId = "",
                Event = ex.Message,
                Details = ex.StackTrace
            };

            await _eventService.PublishEvent(aEvent);
        }

    }

如果我通过编写功能测试以执行该方法来手动执行此方法,或者通过在页面上具有触发该方法执行的按钮,则一切正常,但是当Hangfire尝试执行完全相同的操作时方法,出现此错误。

  

发生一个或多个错误。 (内存不足,无法继续执行程序。)(对象引用未设置为对象的实例。)(对象引用未设置为对象的实例。)(对象引用未设置为对象的实例。) )(对象引用未设置为对象的实例。)(对象引用未设置为对象的实例。)(对象引用未设置为对象的实例。)(对象引用未设置为对象的实例)对象。)(未将对象引用设置为对象的实例。)(未将对象引用设置为对象的实例。)(未将对象引用设置为对象的实例。)(未将对象引用设置为实例)对象引用。)(对象引用未设置为对象的实例。)(对象引用未设置为对象的实例。)(对象引用未设置为对象的实例。)(对象引用未设置为对象的实例。)(对象引用未设置为对象的实例。)(对象引用未设置为实例的实例) (对象引用未设置为对象的实例。)

踪迹是这个。

  

at System.Threading.Tasks.TaskReplicator.Run [TState](ReplicatableUserAction 1 action, ParallelOptions options, Boolean stopOnFirstFailure) at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action 1正文,Action 2 bodyWithState, Func 4 bodyWithLocal,Func 1 localInit, Action 1 localFinally)    ---从之前引发异常的位置开始的堆栈结束跟踪---    在System.Threading.Tasks.Parallel.ThrowSingleCancellationExceptionOrOtherException(ICollection异常,CancellationToken cancelToken,异常otherException)    at System.Threading.Tasks.Parallel.ForWorker [TLocal](Int32 fromInclusive,Int32 toExclusive,ParallelOptions parallelOptions,Action 1 body, Action 2 bodyWithState,Func 4 bodyWithLocal, Func 1 localInit,Action 1 localFinally) at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IList 1列表, ParallelOptions parallelOptions,动作1 body, Action 2 bodyWithState,动作3 bodyWithStateAndIndex, Func 4 bodyWithStateAndLocal,函数5 bodyWithEverything, Func 1 localInit,动作1 localFinally) at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable 1源,ParallelOptions parallelOptions,动作1 body, Action 2 bodyWithState ,动作3 bodyWithStateAndIndex, Func 4 bodyWithStateAndLocal,函数5 bodyWithEverything, Func 1 localInit,动作1 localFinally) at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable 1源,ParallelOptions parallelOptions,动作`1主体)    位于*****。*****。Services.ApiService.SyncProducts(Int32 days,Boolean hasSalesOnly)在C:***************。**** \ Services \ ApiService.cs:第493行

有人可以帮我弄清楚为什么在执行hangfire执行该方法时失败了吗?

在此先感谢所有人。

0 个答案:

没有答案