TPL DataFlow中的排序和线程处理如何工作(寻找一些见解)

时间:2018-08-08 20:20:41

标签: c# parallel-processing task-parallel-library

简介

在为我自己并行化应用程序时,我注意到当过去的某个项目花费很长时间时,最终将只有一个正在运行的线程,其余的线程将等待该项目完成,以便可以返回输出以有序的方式。

这一切似乎很好,因为我只是注释掉了有序属性,并且我的程序全速前进。

我的困惑

我不完全了解的是这些消息在使用TransformBlock1完成但等待该块完成上一个项目时的位置。

在我在此处发布的代码中:

https://github.com/devedse/WpfParallelDataflowInvestigation

问题

关于此代码,我有3个具体问题:

  1. 我已将此块的绑定容量设置为10。但是此绑定容量是否意味着它将在某种输出缓冲区中存储10个项目?
  2. 似乎有限容量也会影响可以排队的项目数量(从一开始就可以看到10个项目排队)。
  3. 我还记录了In:..和Out:..缓冲区计数,但它们始终保持为0。在完成第5项之前,我正在处理的消息在哪里等待输出?

重要代码

public async Task<IEnumerable<string>> ProcessInParallel(IOutputWriter logListener, int maxDegreeOfParallelism = 8)
{
    var theResults = new List<string>();

    var processBlock = new TransformBlock<int, string>(async number =>
    {
        var processedNumber = await ProcessNumber(number);

        return processedNumber;
    }, new ExecutionDataflowBlockOptions()
    {
        MaxDegreeOfParallelism = maxDegreeOfParallelism,
        BoundedCapacity = 10
    });

    var putInListBlock = new ActionBlock<string>(t =>
    {

        if (logListener != null)
        {
            logListener.WriteOutput(t);
        }
        theResults.Add(t);
    }, SynchronizeForUiThread(new ExecutionDataflowBlockOptions()
    {

    }));

    processBlock.LinkTo(putInListBlock, new DataflowLinkOptions() { PropagateCompletion = true });


    var ttt = Task.Run(async () =>
    {
        while (true)
        {
            Console.WriteLine($">>>>> Threads: {currentThreadsRunning}     In: {processBlock.InputCount} Out: {processBlock.OutputCount}      >>    In: {putInListBlock.InputCount} Out: {putInListBlock.Completion.IsCompleted}");
            await Task.Delay(1000);
        }
    });

    var numbers = EnumerateNumberList(100);
    await Task.Run(async () =>
    {
        foreach (var number in numbers)
        {
            Console.WriteLine($"Posting: {number}");
            var result = await processBlock.SendAsync(number);

            if (!result)
            {
                Console.WriteLine("Result is false!!!");
            }

        }
    });



    Console.WriteLine("Completing");
    processBlock.Complete();
    await putInListBlock.Completion;

    return theResults;
}

记录

Posting: 0
>>>>> Threads: 0     In: 0 Out: 0      >>    In: 0 Out: False
Posting: 1
Posting: 2
Posting: 3
Posting: 4
Posting: 5
Posting: 6
Posting: 7
Posting: 8
Posting: 9
Posting: 10
>>>>> Threads: 8     In: 2 Out: 0      >>    In: 0 Out: False
>>>>> Threads: 8     In: 2 Out: 0      >>    In: 0 Out: False
>>>>> Threads: 8     In: 2 Out: 0      >>    In: 0 Out: False
Processed: 4
>>>>> Threads: 8     In: 1 Out: 0      >>    In: 0 Out: False
Processed: 7
>>>>> Threads: 8     In: 0 Out: 0      >>    In: 0 Out: False
>>>>> Threads: 8     In: 0 Out: 0      >>    In: 0 Out: False
Processed: 0
Posting: 11
>>>>> Threads: 8     In: 0 Out: 0      >>    In: 0 Out: False
>>>>> Threads: 8     In: 0 Out: 0      >>    In: 0 Out: False
Processed: 3
Processed: 9
>>>>> Threads: 6     In: 0 Out: 0      >>    In: 0 Out: False
Processed: 8
Processed: 6
Processed: 2
Processed: 1
Posting: 12
Posting: 13
Posting: 14
Posting: 15
>>>>> Threads: 6     In: 0 Out: 0      >>    In: 0 Out: False
>>>>> Threads: 6     In: 0 Out: 0      >>    In: 0 Out: False
Processed: 10
>>>>> Threads: 5     In: 0 Out: 0      >>    In: 0 Out: False
>>>>> Threads: 5     In: 0 Out: 0      >>    In: 0 Out: False
>>>>> Threads: 5     In: 0 Out: 0      >>    In: 0 Out: False
Processed: 13
>>>>> Threads: 4     In: 0 Out: 0      >>    In: 0 Out: False
>>>>> Threads: 4     In: 0 Out: 0      >>    In: 0 Out: False
Processed: 11
Processed: 14
>>>>> Threads: 2     In: 0 Out: 0      >>    In: 0 Out: False
Processed: 12
>>>>> Threads: 1     In: 0 Out: 0      >>    In: 0 Out: False
>>>>> Threads: 1     In: 0 Out: 0      >>    In: 0 Out: False
>>>>> Threads: 1     In: 0 Out: 0      >>    In: 0 Out: False
>>>>> Threads: 1     In: 0 Out: 0      >>    In: 0 Out: False

0 个答案:

没有答案