使用BlockingQueueAgent并行使用asyncSeq

时间:2014-07-25 08:37:07

标签: asynchronous f# parallel-processing async-workflow

由于缺乏对.NET中并发性的理解而再次让自己难堪的时间:P

我正在尝试编写一个可以封装创建异步工作流的函数,该工作流接受asyncSeq输入并将其分发给n个并行使用者。

let doWorkInParallel bufferSize numberOfConsumers consumer input = async {
    let buffer = BlockingQueueAgent<_>(bufferSize)
    let inputFinished = ref false

    let produce seq = async {
        do! seq |> AsyncSeq.iterAsync (Some >> buffer.AsyncAdd)
        do! buffer.AsyncAdd None
    }

    let consumeParallel degree f = async {
        let consume f = async {
            while not <| !inputFinished do
                try
                    let! data = buffer.AsyncGet(waitTimeout)
                    match data with
                    | Some i -> do! f i
                    // whichever consumer gets the end of the input flips the inputFinished ref so they'll all stop processing
                    | None -> inputFinished := true
                with | :? TimeoutException -> ()
        }

        do! [for slot in 1 .. degree -> consume f ]
            |> Async.Parallel
            |> Async.Ignore
    }

    let startProducer = produce input
    let startConsumers = consumeParallel numberOfConsumers consumer

    return! [| startProducer; startConsumers |] |> Async.Parallel |> Async.Ignore
}

并且我一直在使用这样的代码对其进行测试,以模拟快速创建的序列,但每个项目的工作需要一段时间。

let input = seq {1..50} |> AsyncSeq.ofSeq
let action i = async {
    do! Async.Sleep 500
    printfn "%d GET" i
    }
let test = doWorkInParallel 10 2 action input
Async.RunSynchronously test
编辑:我似乎已经解决了我最初的问题(呃,我真的没想过我用EventWaitHandle做了什么,太傻了),但我还是很想知道这是不是用F#来解决这个问题的完全愚蠢的方法。

0 个答案:

没有答案