Question

我希望有一个干净的解决方案，可以在消费者忙着处理时限制特定类型的生产者，而无需编写自己的自定义块。我曾希望下面的代码完全可以做到这一点，但是一旦SendAsync在容量限制命中后阻塞，它的任务就永远不会完成，暗示推迟的消息永远不会消耗掉。

_block = new TransformBlock<int, string>(async i =>
{
    // Send the next request to own input queue
    // before processing this request, or block
    // while pipeline is full. 
    // Do not start processing if pipeline is full!
    await _block.SendAsync(i + 1);
    // Process this request and pass it on to the
    // next block in the pipeline.
    return i.ToString();
}, 
// TransformBlock block has input and output buffers. Limit both, 
// otherwise requests that cannot be passed on to the next 
// block in the pipeline will be cached in this block's output 
// buffer, never throttling this block.
new ExecutionDataflowBlockOptions { BoundedCapacity = 5 });

// This block is linked to the output of the 
// transform block. 
var action = new ActionBlock<string>(async i =>
{
    // Do some very long processing on the transformed element.
    await Task.Delay(1000);
}, 
// Limit buffer size, and consequently throttle previous blocks 
// in the pipeline.
new ExecutionDataflowBlockOptions { BoundedCapacity = 5 });
_block.LinkTo(action);

// Start running.
_block.Post(0);

我想知道链接的ActionBlock是否有任何理由不使用推迟的消息。

Answer 1

我遇到了和你一样的问题。我没有深入研究LinkTo的实现，但我认为只有当源块收到一些时才会传播消息。我的意思是，可能存在源块在其输入中有一些消息的情况，但是在它接收到下一个Post / SendAsync之前它不会处理它们。这就是你的情况。

这是我的解决方案，它为我工作。

首先声明＆＃34;引擎＆＃34;

/// <summary>
/// Engine-class (like a car engine) that produced a lot count (or infinite) of actions.
/// </summary>
public class Engine
{
    private BufferBlock<int> _bufferBlock;

    /// <summary>
    /// Creates source block that produced stub data.
    /// </summary>
    /// <param name="count">Count of actions. If count = 0 then it's infinite loop.</param>
    /// <param name="boundedCapacity">Bounded capacity (throttling).</param>
    /// <param name="cancellationToken">Cancellation token (used to stop infinite loop).</param>
    /// <returns>Source block that constantly produced 0-value.</returns>
    public ISourceBlock<int> CreateEngine(int count, int boundedCapacity, CancellationToken cancellationToken)
    {
        _bufferBlock = new BufferBlock<int>(new DataflowBlockOptions { BoundedCapacity = boundedCapacity });

        Task.Run(async () =>
        {
            var counter = 0;
            while (count == 0 || counter < count)
            {
                await _bufferBlock.SendAsync(0);
                if (cancellationToken.IsCancellationRequested)
                    return;
                counter++;
            }
        }, cancellationToken).ContinueWith((task) =>
        {
            _bufferBlock.Complete();
        });

        return _bufferBlock;
    }
}

然后是使用引擎的生产者

/// <summary>
/// Producer that generates random byte blobs with specified size.
/// </summary>
public class Producer
{
    private static Random random = new Random();

    /// <summary>
    /// Returns source block that produced byte arrays. 
    /// </summary>
    /// <param name="blobSize">Size of byte arrays.</param>
    /// <param name="count">Total count of blobs (if 0 then infinite).</param>
    /// <param name="boundedCapacity">Bounded capacity (throttling).</param>
    /// <param name="cancellationToken">Cancellation token (used to stop infinite loop).</param>
    /// <returns>Source block.</returns>
    public static ISourceBlock<byte[]> BlobsSourceBlock(int blobSize, int count, int boundedCapacity, CancellationToken cancellationToken)
    {
        // Creating engine with specified bounded capacity.
        var engine = new Engine().CreateEngine(count, boundedCapacity, cancellationToken);

        // Creating transform block that uses our driver as a source.
        var block = new TransformBlock<int, byte[]>(
            // Useful work.
            i => CreateBlob(blobSize),
            new ExecutionDataflowBlockOptions
            {
                // Here you can specify your own throttling. 
                BoundedCapacity = boundedCapacity,
                MaxDegreeOfParallelism = Environment.ProcessorCount,
            });
        // Linking engine (and engine is already working at that time).
        engine.LinkTo(block, new DataflowLinkOptions { PropagateCompletion = true });
        return block;
    }

    /// <summary>
    /// Simple random byte[] generator.
    /// </summary>
    /// <param name="size">Array size.</param>
    /// <returns>byte[]</returns>
    private static byte[] CreateBlob(int size)
    {
        var buffer = new byte[size];
        random.NextBytes(buffer);
        return buffer;
    }
}

现在您可以将生产者与消费者一起使用（例如ActionBlock）

        var blobsProducer = BlobsProducer.CreateAndStartBlobsSourceBlock(0, 1024 * 1024, 10, cancellationTokenSource.Token);

        var md5Hash = MD5.Create();

        var actionBlock = new ActionBlock<byte[]>(b => 
        {
            Console.WriteLine(GetMd5Hash(md5Hash, b));
        },
        new ExecutionDataflowBlockOptions() { BoundedCapacity = 10 });

        blobsProducer.LinkTo(actionBlock);

希望它会对你有所帮助！

链接块时，TPL数据流SendAsync任务永远不会完成

1 个答案: