Question

我有一个应用程序，我最初在其中建立了在我所属的所有团队中我是多少（Microsoft）团队的所有者。
因此，如果我是 37 个团队的成员，我最终需要列出我实际拥有的 13 个团队。

它有效 - 为每个团队查询所有者的 MS Graph - 然而，一些用户拥有数百个团队，很明显，当不得不等待顺序加载时，加载时间是不可接受的。所以我试图用 Task.Select 和 Task.WhenAll 解决这个问题。然而，任务是按顺序运行的，而不是并行的。
我很想将总加载时间降低到大约 250 毫秒，而不是 250 乘以 37。我已经读过，如果我在任务中使用 task.WhenAll 会导致 .Result 被冒犯，导致它按顺序运行，但我不知道如何使它在并行线程中运行。

 private static async Task DispatchGetTeamOwnersAsync(JEnumerable<JToken> userTeams, GraphittiBox.Model.TokenObject token)
        {

            var tasks = userTeams.Select(async team =>
            {
                Team t = JsonConvert.DeserializeObject<Team>(team.ToString());
                Stopwatch clock = Stopwatch.StartNew();
                LogService.WriteLog("await: GetOwnersOfTeamAsync");

                
                HttpClient httpClient = new HttpClient();
                httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(token.Token_type, token.Access_token);
                var url = new Uri("https://graph.microsoft.com/v1.0/groups/{groupId}/owners?$select=mail,id,displayName".Replace("{groupId}", t.Id));
                var response = httpClient.GetAsync(url);

                var content = response.Result.Content.ReadAsStringAsync();

                JObject owners = JsonConvert.DeserializeObject<JObject>(content.Result);

                JsonOwnersCollection.Add(new OwnersAsyncList(t.Id, owners));


                clock.Stop();
                LogService.WriteLog("Done (" + clock.ElapsedMilliseconds.ToString() + " ms)");
            });

            await Task.WhenAll(tasks);

        }

日志文件：

05-03-2021 08:21:04 Info    User initiated 'ProcessForm'
05-03-2021 08:21:05 Info    User has a total of 37 UserJoinedTeams
05-03-2021 08:21:05 Info    Method: GetAsyncOwners (ProcessForm)
05-03-2021 08:21:05 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:06 Info    Done (335 ms)
05-03-2021 08:21:06 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:06 Info    Done (237 ms)
05-03-2021 08:21:06 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:06 Info    Done (231 ms)
05-03-2021 08:21:06 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:07 Info    Done (214 ms)
05-03-2021 08:21:07 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:07 Info    Done (219 ms)
05-03-2021 08:21:07 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:07 Info    Done (229 ms)
05-03-2021 08:21:07 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:07 Info    Done (217 ms)
05-03-2021 08:21:07 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:08 Info    Done (314 ms)
05-03-2021 08:21:08 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:08 Info    Done (225 ms)
05-03-2021 08:21:08 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:08 Info    Done (203 ms)
05-03-2021 08:21:08 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:08 Info    Done (206 ms)
05-03-2021 08:21:08 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:08 Info    Done (251 ms)
05-03-2021 08:21:08 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:09 Info    Done (289 ms)
05-03-2021 08:21:09 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:09 Info    Done (224 ms)
05-03-2021 08:21:09 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:09 Info    Done (258 ms)
05-03-2021 08:21:09 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:10 Info    Done (240 ms)
05-03-2021 08:21:10 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:10 Info    Done (317 ms)
05-03-2021 08:21:10 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:10 Info    Done (297 ms)
05-03-2021 08:21:10 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:10 Info    Done (255 ms)
05-03-2021 08:21:10 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:11 Info    Done (208 ms)
05-03-2021 08:21:11 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:11 Info    Done (243 ms)
05-03-2021 08:21:11 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:11 Info    Done (260 ms)
05-03-2021 08:21:11 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:12 Info    Done (369 ms)
05-03-2021 08:21:12 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:12 Info    Done (248 ms)
05-03-2021 08:21:12 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:12 Info    Done (232 ms)
05-03-2021 08:21:12 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:12 Info    Done (300 ms)
05-03-2021 08:21:12 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:13 Info    Done (233 ms)
05-03-2021 08:21:13 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:13 Info    Done (249 ms)
05-03-2021 08:21:13 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:13 Info    Done (253 ms)
05-03-2021 08:21:13 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:13 Info    Done (262 ms)
05-03-2021 08:21:13 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:14 Info    Done (243 ms)
05-03-2021 08:21:14 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:14 Info    Done (227 ms)
05-03-2021 08:21:14 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:14 Info    Done (235 ms)
05-03-2021 08:21:14 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:14 Info    Done (257 ms)
05-03-2021 08:21:14 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:15 Info    Done (225 ms)
05-03-2021 08:21:15 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:15 Info    Done (231 ms)
05-03-2021 08:21:15 Info    await: GetOwnersOfTeamAsync
05-03-2021 08:21:15 Info    Done (227 ms)
05-03-2021 08:21:15 Info    User is owner of 13 UserJoinedTeams

Answer 1

您正在使用

var content = response.Result.Content.ReadAsStringAsync();

.Result 部分将导致阻塞，直到任务完成。你应该等待任务。

Answer 2

所有这些都可以简化为：

var ownerTasks=userTeams
       .Cast<JObject>()
       .Select(j=>{
           var id=j.GetValue("Id");
           return $"https://graph.microsoft.com/v1.0/groups/{id}/owners?$select=mail,id,displayName";
        })
        .Select(async url=>await httpClient.GetStringAsync(url));
        .Select(json=>JObject.Parse(json));

var owners=await Task.WhenAll(ownerTasks);

如果您使用 Microsoft Graph SDK 或 OData 客户端，则可以完全避免使用 JObject

Task.WhenAll 不执行任务，而是等待任务完成。

这段代码有几个问题

创建一个新的 HttpClient 实例而不是重用一个实例是一个严重的错误，会导致套接字耗尽
代码使用 .Result 阻止异步调用。这可确保一次只能调用一个 GetAsync() 或 ReadAsStringAsync。
将结果添加到全局集合 JsonOwnersCollection。这需要锁定或并发集合。虽然不需要

Select 是同步的，并且只返回 Task，因为（否则无用）async 关键字。我确定编译器生成了一个警告，说明了这一点。

代码至少应该是这样的：

//Use one instance only. It's thread-safe and *meant* to be reused
var httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(token.Token_type, token.Access_token);

var ownerTasks = userTeams
    .Select(async team =>
    {
        //What's the point of this? 
        Team t = JsonConvert.DeserializeObject<Team>(team.ToString());
        var clock = Stopwatch.StartNew();
        LogService.WriteLog("await: GetOwnersOfTeamAsync");
        var url = new Uri("https://graph.microsoft.com/v1.0/groups/{groupId}/owners?$select=mail,id,displayName".Replace("{groupId}", t.Id));

        var response = await httpClient.GetAsync(url);
        var content = await response.Content.ReadAsStringAsync();

        var owners = JsonConvert.DeserializeObject<JObject>(content);
        LogService.WriteLog($"Done ({clock.ElapsedMilliseconds} ms)");

        return new OwnersAsyncList(t.Id, owners);         
    });

var owners=await Task.WhenAll(ownerTasks);

还有其他可以改变的事情。这两行：

var response = await httpClient.GetAsync(url);
var content = await response.Content.ReadAsStringAsync();

可以替换为

var content=await httpClient.GetStringAsync(url);

URL 可以一步构建，避免临时字符串：

var url = new Uri($"https://graph.microsoft.com/v1.0/groups/{t.Id}/owners?$select=mail,id,displayName");

而不是将 team 转换为 string 只是为了将其反序列化为 Team - 为什么不传递 Team 对象的列表呢？即使有理由使用 JSToken 也没有理由序列化和解析它。 JObject 是 JToken。如果改为传递 JArray 或 JProperty，序列化/反序列化将失败。

可以通过以下方式获取 ID：

var id=((JObject)team).GetValue("Id");

这一切都变成了：

var ownerTasks=userTeams
       .Cast<JObject>()
       .Select(j=>{
           var id=j.GetValue("Id");
           return $"https://graph.microsoft.com/v1.0/groups/{id}/owners?$select=mail,id,displayName";
        })
        .Select(async url=>await httpClient.GetStringAsync(url));
        .Select(json=>JObject.Parse(json));

var owners=await Task.WhenAll(ownerTasks);

最后，HTTP 端点的主要问题不是如何进行多个并发调用。一个简单的 urls.Select(url=>httpClient.GetStringAsync(url)) 就足够了。这是如何限制调用，并且一次只执行几个。

一种简单的方法是使用 Dataflow 类以有限的并行度执行多个并发操作。

var dop=new ExecutionDataflowBlockOptions 
{
    MaxDegreeOfParallelism=8 //Adjust based on Graph's throttling limits
};
var block=new TransformBlock<string,JObject>(async j=>{
     var id=j.GetValue("Id");
     var url= $"https://graph.microsoft.com/v1.0/groups/{id}/owners?$select=mail,id,displayName";
     var json=await httpClient.GetStringAsync(url));
      return JObject.Parse(json)
    })

var buffer=new BufferBlock<JObject>();

block.LinkTo(buffer)

foreach(var j in teams)
{
    block.Post(j);
}

block.Complete();
//Wait for all operations to complete
await block.Completion;
//The buffer contains all results now

Answer 3

此处使用 var 隐藏了您的中间体是 Task<T> 而不是 T 的事实。应该避免使用 .Result，它会阻塞。

//var response = httpClient.GetAsync(url);
//var content = response.Result.Content.ReadAsStringAsync();
//JObject owners = JsonConvert.DeserializeObject<JObject>(content.Result);

var response = await httpClient.GetAsync(url);  
var content = await response.Content.ReadAsStringAsync();
JObject owners = JsonConvert.DeserializeObject<JObject>(content);

请注意，您的原始循环代码没有任何 await，这就是为什么您的任务是同步的，而 WhenAll 按顺序运行它们。

Task.WhenAll 多线程顺序运行而不是并行运行

3 个答案: