PLINQ查询给出溢出异常

时间:2012-10-30 02:46:03

标签: c# linq task-parallel-library plinq

我正在运行PLINQ查询,如下所示:

ParallelQuery<string> winningCombos = from n in nextComboMaker.GetNextCombo()
                                              .AsParallel().WithCancellation(_cancelSource.Token)
                                              where ComboWasAWinner(n) 
                                              select n;

ConcurrentBag<string> wins = new ConcurrentBag<string>();

foreach (var winningCombo in winningCombos)
{
        wins.Add(winningCombo);
        if (wins.Count == _maxWinsAllowed)
            break;
}

GetNextCombo方法只返回下一个字母和数字组合,可能达到数十亿/万亿。

现在当我选择一个大于Int32允许大小的组合范围时抛出异常,当它运行的组合计数器是2147483584时它总是抛出。

我已经确定GetNextCombo中没有任何东西可以通过创建一个假组合来每次返回(做一个收益率返回“234gf24fa23 ......”等)。

LINQ抛出异常:

System.AggregateException was unhandled by user code
  Message=One or more errors occurred.
  Source=System.Core
  StackTrace:
       at System.Linq.Parallel.QueryTaskGroupState.QueryEnd(Boolean userInitiatedDispose)
       at System.Linq.Parallel.MergeExecutor`1.Execute[TKey](PartitionedStream`2 partitions, Boolean ignoreOutput, ParallelMergeOptions options, TaskScheduler taskScheduler, Boolean isOrdered, CancellationState cancellationState, Int32 queryId)
       at System.Linq.Parallel.PartitionedStreamMerger`1.Receive[TKey](PartitionedStream`2 partitionedStream)
       at System.Linq.Parallel.ForAllOperator`1.WrapPartitionedStream[TKey](PartitionedStream`2 inputStream, IPartitionedStreamRecipient`1 recipient, Boolean preferStriping, QuerySettings settings)
       at System.Linq.Parallel.UnaryQueryOperator`2.UnaryQueryOperatorResults.ChildResultsRecipient.Receive[TKey](PartitionedStream`2 inputStream)
       at System.Linq.Parallel.WhereQueryOperator`1.WrapPartitionedStream[TKey](PartitionedStream`2 inputStream, IPartitionedStreamRecipient`1 recipient, Boolean preferStriping, QuerySettings settings)
       at System.Linq.Parallel.UnaryQueryOperator`2.UnaryQueryOperatorResults.ChildResultsRecipient.Receive[TKey](PartitionedStream`2 inputStream)
       at System.Linq.Parallel.ScanQueryOperator`1.ScanEnumerableQueryOperatorResults.GivePartitionedStream(IPartitionedStreamRecipient`1 recipient)
       at System.Linq.Parallel.UnaryQueryOperator`2.UnaryQueryOperatorResults.GivePartitionedStream(IPartitionedStreamRecipient`1 recipient)
       at System.Linq.Parallel.UnaryQueryOperator`2.UnaryQueryOperatorResults.GivePartitionedStream(IPartitionedStreamRecipient`1 recipient)
       at System.Linq.Parallel.QueryOperator`1.GetOpenedEnumerator(Nullable`1 mergeOptions, Boolean suppressOrder, Boolean forEffect, QuerySettings querySettings)
       at System.Linq.Parallel.ForAllOperator`1.RunSynchronously()
       at StockWiz.Library.PLINQArrayProcessor.DoProcessing() in C:\Users\dad\Documents\BitBucket\stockwiz_clone\stockwiz\StockWiz.Library\PLINQArrayProcessor.cs:line 50
       at System.Threading.Tasks.Task.Execute()
  InnerException: System.OverflowException
       Message=Arithmetic operation resulted in an overflow.
       Source=System.Core
       StackTrace:
            at System.Linq.Parallel.PartitionedDataSource`1.ContiguousChunkLazyEnumerator.MoveNext(T& currentElement, Int32& currentKey)
            at System.Linq.Parallel.WhereQueryOperator`1.WhereQueryOperatorEnumerator`1.MoveNext(TInputOutput& currentElement, TKey& currentKey)
            at System.Linq.Parallel.ForAllOperator`1.ForAllEnumerator`1.MoveNext(TInput& currentElement, Int32& currentKey)
            at System.Linq.Parallel.ForAllSpoolingTask`2.SpoolingWork()
            at System.Linq.Parallel.SpoolingTaskBase.Work()
            at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
            at System.Threading.Tasks.Task.Execute()
       InnerException: 

我想知道是否有任何我可以做的事情来改变这个查询不溢出,我做任何事情的顺序等等。也许没有linq查询做的地方和选择,虽然我试过这个:

 var query = nextComboMaker.GetNextCombo().AsParallel();

 query.ForAll(x => if(ComboWasAWinner(x) wins.Add(x) );

仍然是一样的溢出。

1 个答案:

答案 0 :(得分:6)

您应该可以使用自定义分区程序解决此问题。在这种情况下,PLINQ选择的默认分区程序仅支持int-range项目数。

有关如何执行此操作,请参阅http://msdn.microsoft.com/en-us/library/dd997416.aspx