Question

我正在使用Parallel.ForEach来处理一堆项目。问题是，我想根据打开的工作人员（插槽）的数量来确定哪些项目可以正常工作。例如。如果我正在工作8个并行的东西，并在任务1-4之间打开一个插槽，我想为这些插槽分配简单的工作。插槽的下半部分将得到很大的努力。通过这种方式，我不会将所有8个插槽用于执行硬/长时间工作，将首先运行简单/快速项目。我已经实现了如下：

守则

const int workers = 8;
List<Thing> thingsToDo = ...; //Get the things that need to be done.
Thing[] currentlyWorkingThings = new Thing[workers]; //One slot for each worker.

void Run() {
    Parallel.ForEach(PrioritizeThings(thingsToDo), o => {
        int index = 0;

        //"PrioritizeTasks" added this thing to the list of currentlyWorkingThings.
        //Find my position in this list.
        lock (currentlyWorkingThings)
            index = currentlyWorkingThings.IndexOf(o);

        //Do work on this thing...

        //Then remove it from the list of currently working things, thereby
        //  opening a new slot when this worker returns/finishes.
        lock (currentlyWorkingThings)
            currentlyWorkingThings[index] = null;
    });
}

IEnumerable<Thing> PrioritizeThings(List<Thing> thingsToDo) {
    int slots = workers;
    int halfSlots = (int)Math.Ceiling(slots / 2f);

    //Sort thingsToDo by their difficulty, easiest first.

    //Loop until we've worked every Thing.
    while (thingsToDo.Count > 0) {
        int slotToFill = ...; //Find the first open slot.
        Thing nextThing = null;

        lock (currentlyWorkingThings) {
            //If the slot is in the "top half", get the next easy thing - otherwise
            //  get the next hard thing.
            if (slotToFill < halfSlots)
                nextThing = thingsToDo.First();
            else
                nextThing = thingsToDo.Last();

            //Add the nextThing to the list of currentlyWorkingThings and remove it from
            //  the list of thingsToDo.
            currentlyWorkingThings[slotToFill] = nextThing;
            thingsToDo.Remove(nextThing);
        }

        //Return the nextThing to work.
        yield return nextThing;
    }
}

问题

所以我在这里看到的问题是Parallel在一个插槽打开之前（在现有的事情完成之前）要求PrioritizeThings处理下一件事。我假设Parallel正在展望未来并提前准备好工作。我希望不要这样做，只有在完成后才填充工人/插槽。我想到解决这个问题的唯一方法就是在PrioritizeThings中添加一个睡眠/等待循环，在它看到一个合法的开放槽之前不会返回任何东西。但是我不喜欢那样，我希望有一些方法可以让Parallel在上班前等待更长时间。有什么建议吗？

Answer 1

有一种方法（有点）可以完全支持你所描述的情况。

创建ForEach时，您需要使用非标准ParallelOptions传递TaskScheduler。困难的部分是创建一个TaskSchedueler为您做优先级系统，幸运的是，Microsoft发布了一组示例，其中包含一个名为“ParallelExtensionsExtras”的调度程序及其调度程序QueuedTaskScheduler

private static void Main(string[] args)
{
    int totalMaxConcurrancy = Environment.ProcessorCount;
    int highPriorityMaxConcurrancy = totalMaxConcurrancy / 2;

    if (highPriorityMaxConcurrancy == 0)
        highPriorityMaxConcurrancy = 1;

    QueuedTaskScheduler qts = new QueuedTaskScheduler(TaskScheduler.Default, totalMaxConcurrancy);
    var highPriortiyScheduler = qts.ActivateNewQueue(0);
    var lowPriorityScheduler = qts.ActivateNewQueue(1);

    BlockingCollection<Foo> highPriorityWork = new BlockingCollection<Foo>();
    BlockingCollection<Foo> lowPriorityWork = new BlockingCollection<Foo>();

    List<Task> processors = new List<Task>(2);

    processors.Add(Task.Factory.StartNew(() =>
    {
        Parallel.ForEach(highPriorityWork.GetConsumingPartitioner(),  //.GetConsumingPartitioner() is also from ParallelExtensionExtras, it gives better performance than .GetConsumingEnumerable() with Parallel.ForEeach(
                         new ParallelOptions() { TaskScheduler = highPriortiyScheduler, MaxDegreeOfParallelism = highPriorityMaxConcurrancy }, 
                         ProcessWork);
    }, TaskCreationOptions.LongRunning));

    processors.Add(Task.Factory.StartNew(() =>
    {
        Parallel.ForEach(lowPriorityWork.GetConsumingPartitioner(), 
                         new ParallelOptions() { TaskScheduler = lowPriorityScheduler}, 
                         ProcessWork);
    }, TaskCreationOptions.LongRunning));


    //Add some work to do here to the highPriorityWork or lowPriorityWork collections


    //Lets the blocking collections know we are no-longer going to be adding new items so it will break out of the `ForEach` once it has finished the pending work.
    highPriorityWork.CompleteAdding();
    lowPriorityWork.CompleteAdding();

    //Waits for the two collections to compleatly empty before continueing
    Task.WaitAll(processors.ToArray());
}

private static void ProcessWork(Foo work)
{
    //...
}

即使您有两个Parallel.ForEach运行实例，但两个实例的总和不会超过您为MaxConcurrency传入QueuedTaskScheduler构造函数的值如果两者都有工作要做，将优先清空highPriorityWork集合（最多可达到所有可用插槽的1/2，这样你就不会阻塞低优先级队列，你可以根据您的性能需求，轻松将其调整为更高或更低的比率。

如果你不希望高优先级总是赢，你宁愿有一个“循环”式调度程序在两个列表之间交替（所以你不希望快速项目总是赢，但只是你可以将它们放入缓慢的项目中）你可以将相同的优先级设置为两个或更多个队列（或者只使用RoundRobinTaskSchedulerQueue执行相同的操作）

使Parallel.ForEach等待工作直到插槽打开

1 个答案: