TPL数据流与铁路规划

时间:2016-06-09 00:46:18

标签: c# task-parallel-library tpl-dataflow

我正在尝试使用铁路编程实现一个中等复杂的TPL数据流管道,如果它们在当前块中成功,则消息可以在“快乐路径”下发送,如果它们不成功则转到故障块。我遇到的问题是,完成意外地继续传播。我没有在本地运行的问题,但在VM上运行它会导致问题。

填充matcherBuffer后,我将完成发送到“Head”或matchBuffer。

    private void CreatePipeline()
    {
        int blockCapacity;
        if(!Int32.TryParse(ConfigurationManager.AppSettings["TPLBlockCapacity"], out blockCapacity)){
            logger.WriteInformation("TPLBlockCapacity app.config value could not be properly parsed.  TPLBlockCapacity will default to 1000.");
            blockCapacity=1000;
        }
        var linkOptions = new DataflowLinkOptions { PropagateCompletion = true };
        var blockOptions = new ExecutionDataflowBlockOptions { BoundedCapacity = blockCapacity };
        matcher = new Matcher(secondarySource, logger);
        comparer = new Comparer(logger, report, fileSystem, dateTime);
        confirmer = new Confirmer(confirmationSource, logger, dateTime, report, fileSystem);
        changer = new Changer(logger, rioEmployeeEditor);
        unconfirmedWriter = new UnconfirmedWriter(logger, rioEmployeeEditor);
        failer = new Failer(logger, report, fileSystem, dateTime);
        finalizer = new Finalizer(logger, fileSystem, archiveFactory);


        var matchBuffer = new BufferBlock<IResult>(blockOptions);
        var matcherBlock = new TransformBlock<IResult, IResult>(result => matcher.Process(result));
        var failureBlock = new TransformBlock<IResult, IResult>(result => failer.Process(result));
        var comparerBlock = new TransformBlock<IResult, IResult>(result => comparer.Process(result));
        var confirmerBlock = new TransformBlock<IResult, IResult>(result => confirmer.Process(result));
        var confirmBroadcaster = new BroadcastBlock<IResult>(x => x.Clone());
        var unconfirmedBlock = new ActionBlock<IResult>(result => unconfirmedWriter.Process(result));
        var changerBlock = new TransformBlock<IResult, IResult>(result => changer.Process(result));
        var finalizerBlock = new ActionBlock<IResult>(result => finalizer.Process(result));
        //Message buffer from Selector links
        matchBuffer.LinkTo(matcherBlock, linkOptions, result => result.Success);
        //Matcher links
        matcherBlock.LinkTo(comparerBlock, linkOptions, result => result.Success);
        matcherBlock.LinkTo(failureBlock, result => !result.Success);
        //Comparer links

        comparerBlock.LinkTo(confirmerBlock, linkOptions, result => result.Success);
        comparerBlock.LinkTo(failureBlock, result => !result.Success);
        //Confirmer links

        confirmerBlock.LinkTo(confirmBroadcaster, linkOptions, result => result.Success);
        confirmerBlock.LinkTo(failureBlock, result => !result.Success);
        //ConfirmBroadcaster links
        confirmBroadcaster.LinkTo(changerBlock, result => result.Success);
        confirmBroadcaster.LinkTo(unconfirmedBlock, linkOptions, result => result.Success);

        changerBlock.LinkTo(finalizerBlock, linkOptions);
        failureBlock.LinkTo(finalizerBlock);

        unconfirmedBlock.Completion.ContinueWith(_ =>
        {
            Console.WriteLine("Unconfirmed Block Complete.");
            failureBlock.Complete();

        });
        failureBlock.Completion.ContinueWith(_ =>
        {
            Console.WriteLine("Failure Block Complete.");
            changerBlock.Complete();

        });
        Head = matchBuffer;
        Tail = finalizerBlock;
        ErrorBlock = failureBlock;
    }

我能想到的每个场景在本地运行良好,所有块都可以通过终结器获得我的预期结果,但是当我推送到我们的VM时,我会得到完全不同的行为。应用程序提前结束,终结器甚至没有开始执行。所有这些块都可以花费任何时间来完成,但这似乎不是本地问题。我可以在任何这些块中放置一个Thread.Sleep,它仍然按预期执行,但我仍然在VM上过早关闭。

要查看我的预期流程,请将以下内容保存到XML文件并在www.draw.io中打开。黑线是数据流,红线是传播流。

    <mxfile userAgent="Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36" version="5.5.1.6" editor="www.draw.io" type="device"><diagram>3Vtds6I4EP01Ps4UCKj3cbxzZ/Zhp2qr3K3ZfcxohOwgsWK8H/PrJ5FuPoIoKISrvkiaJoTu032aJIy8x83rV0G20Te+ovFo7KxeR97n0XjsTqa++tOSt1QynTmpIBRsBUq5YMF+URCi2p6t6K6kKDmPJduWhUueJHQpSzIiBH8pq615XL7rloR4x1ywWJK4Kv3OVjJKpbPxJJf/QVkY4Z3dyUN65gdZ/gwF3ydwv9HYWx9+6ekNwb7gQXcRWfGXgsh7UnYVnKue9dHm9ZHG2rZotvS6LzVns3ELmsDYzlwwg3HIN3x2ulKmgCYXMuIhT0j8lEvnh+ejugdHtSK5idWhqw7pK5P/avHHAFr/wZn/qZRv4Geyl1yJ8r7/5HwLetXxwyPt+F4sYYSqfUAEESEFLS8V6bEXLoNn/kr5hkrxphQEjYlkz2U3E0BLmOnlFlMHYLTjBoSxPJN4D50uaKwgyUXFsC8Rk3SxJYfHeFGBUzZe7aM/UyEphtbxx4KzE0AVRF0AzZccwi4CLyrAFy+7xg4uBEDfQFLGEW+FU7qpz+kOOgUZIKoIMkhq9kHm9xOlBdM6H8eZObNzmW0Ppm5q3W1MWPJBqDu/FzMfLv0kBNFXocKWs0TuCj3/pQW1seQCg2WJ1FD3r1LHdo6HdLg5OrLnbgSYKVr6OGASnpxESFNXZ3DJGjlczsOg1/x93OwepkSwO5Yl2EM6JLjolPtMd5spNIV4paMLXAmWKRDMNyKXEVX84sz367U+aEc0axbHjzxWDKW1sT7x5jsp+E9aOOMcfr1Qk+tZ5KYs79wLN0GGLAYQVKZ9B1DmI0xcD80iqD77XhMa8NDd+hWP2zPfKf+38avhxLKj3WkvjjUz2mx6rWOb3adr5vP7qUMrmLBfKVkL+isrJWw3LX3aqfuKkGrxcm5oHmavM6C+BHf1NH177JzNw1hhZ6hw7oadISqLgQrvb7bZ2cdydxB2Duyw85B+Tcm4f8cOxc4ns+0FmAggTd4fO1sL+ivZ2TcnhM/QbTv1IEiD/iJ29hu+jV+AO/BOgZ0f+UZ79Cbp2cPot0LPvaTxAekZwrIYqfDOYH9i14Plg7vhyCPGvXeOPJnzLuFIsNf9caSNyOuCIwNATlPSa6c+6XjGA6xa4rZkzcTmNsktQMKxQm7dziDmWXbafZot5VTgjYHjxoXXgwza5Ug4U/5Vrjbrv5pFlEpHmOhxswn228MsTy98jXAZohqCBFwCl6XFuGyWqaXfWqPUWPSbQoapg1NlXGX96wnewmq+UfTli/ttQdSO4Y+h6X2kKoNlJu1mnQ31M4kN1zb7z0dg7yr368VhwclqSXbyNuuAic05aNy+Z+8l11bNjeE3xBuZh4QOLn3oKcEb/D97OJ3fPQNpbfUfynt7ruYDdFEhiv9J1M7WQw2vvOx8FypabyOITepEBNgIYs/SXlZrtRkWYqXNcWAo28WZ6yCrDbGWNOlnTuRk3X35K1zLDD2kl819bIN6GS1RLGcikoTtc18Xmcx8a7CaymzXI0c2cHabyiyVHGYt4DqmOzpcVOp0MqDbPben3gtbOWyQiQFzQqdhUqpWbkZHWLmdm2FqvSRgkKVRIZ7Vn0G7s4oSnFdIpF8IiwfJo5WXAJtfpSC9FQ3BVDypMHsPnOJmD27FGP1svoDJJUwxXU081Zq77+yDO5rrdjg3nZU+1093RDSt1k0Lqr5AHDt6l0NMJeOJaixYqEw8Gk9iZdD5ij2rw1AfouiHiomyRN25pGfAR8FeywuoEHTHfpEfBwXtbe1hJdFNfVpFXpio45iu9YU6cpj66vITiKX2/nynQpAl4d8HKHzwNa+l+VfdM5iPgs/dBGO2OQ6D0T+y8IQ6xVg02ahBLKpm/k1n6uH8w1nv6Tc=</diagram></mxfile>

编辑:调用填充缓冲区的代码:

            private static void Start(){
    // ...
    var selectionStrategy=new SelectionStrategyFactory().GetSelectionStrategy(selectionStrategyType);
    var selector=new Selector(selectionStrategy, logger, dateTime);
    selector.LinkTo(Pipeline.Head, Pipeline.ErrorBlock);
    selector.Process(); //fill the Queue Here
            Pipeline.Head.Complete();
    Pipeline.Tail.WaitForCompletion();
    }

实际填充缓冲区:

        class Selector{
    public void Process(){
        try{
        do{
            foreach(var employee in selectionStrategy.Select()){
            var result = new Result();
            target.Post(result);  //Changing this to target.SendAsync fixed the issue
            }
        } while(selectionStrategy.Stop()==false);
        }
        catch(Exception ex){
        //..
        }
        finally{
        //..
        }
    }
    }

1 个答案:

答案 0 :(得分:0)

很抱歉没有发布其他代码。我终于弄清楚了我的问题。

我通过执行target.Post(IResult)来填充matchBuffer(目标)。事实证明,我需要target.SendAsync(IResult)。我只能假设我的管道完成在目标接受消息之前传播,但(显然)在我的所有消息发送之后。