Question

我正在使用link上此博客文章中所述的ForEachAsync循环。

我的循环体执行I / O（异步/等待）和CPU绑定操作的组合。在这种情况下，确定并行度需要考虑哪些因素？

public static Task ForEachAsync<T>(
    this IEnumerable<T> source,
    int degreeOfParallelism,
    Func<T, Task> body)
{
    if (degreeOfParallelism <= 0)
    {
        throw new ArgumentOutOfRangeException(nameof(degreeOfParallelism));
    }

    return Task.WhenAll(
        from partition in Partitioner
                          .Create(source)
                          .GetPartitions(degreeOfParallelism)
        select Task.Run(async () =>
        {
            using (partition)
            {
                while (partition.MoveNext())
                {
                    await body(partition.Current);
                }
            }
        }));
}

public async Task ProcessChangesAsync(
    IChangeFeedObserverContext context,
    IReadOnlyList<Document> docs,
    CancellationToken cancellationToken)
{
    // This function receives list of documents that need to be processed in parallel
    await docs.ForEachAsync(8, async doc =>
    {
        await ProcessDocument(doc);
    });
}

public async Task ProcessDocument(Document doc)
{
    // We get other details for this entity from database.
    var dbResult = await repository.GetDetailsFromDB(doc);

    // Get some more data from external services
    var moreDetails = await httpClient.GetMoreDataFromOtherServices(dbResult);

    // The data fetched above consists of list of elements and we iterate over the list and run business logic.
    // Use data mappers to convert the data back to entities. These are all CPU bound operations

    // Then finally we persist details in DB
    await repository.WriteToDB(...);
}

Answer 1

除非您想限制正在运行的任务的数量，为什么不让系统和工作本身按其方式工作？

public static Task ForEachAsync<T>(this IEnumerable<T> source, Func<T, Task> body)
    => Task.WhenAll(from item in source select body(item));

如何确定由CPU和I / O绑定的操作组成的循环体中的并行度？

1 个答案: