如何为HttpClient设置最大任务限制

时间:2019-05-07 20:12:56

标签: c# task httpclient

向网站发送多个请求时,我收到“ 429 Too Many Request”。我希望能够发送多个请求以从URL获得响应(有限制)或批量运行任务,这样我就不会收到来自网站的错误。

下面是我累的代码。它工作正常,但是在发送了少量请求后,却给出了过多的请求消息。

const int maxConcurrency = 15;
try
{
    List<Task<HttpResponseMessage>> taskList = new List<Task<HttpResponseMessage>>();
    List<string> replies = new List<string>();

    for (int i = 0; i < lbURLToSearch.Items.Count; i++)
    {

        taskList.Add(reusableHTTPClient.GetAsync(lbURLToSearch.Items[i].ToString()));
        if (taskList.Count >= maxConcurrency)
            Task.WaitAny(taskList.ToArray());
    }

    var httpClientResponse = await Task.WhenAll(taskList);

    foreach (var htmlContent in httpClientResponse)
    {
        using (var reader = new StreamReader(await htmlContent.Content.ReadAsStreamAsync()))
        {
            //replies.Add(JsonConvert.DeserializeObject<string>(reader.ReadToEnd()));
            rawHTMLFromSource += await reader.ReadToEndAsync();
        }
    }

    foreach (var reply in replies)
    {
        // TODO:        
    }
}
catch
{

}

如何减少请求数量?

0 个答案:

没有答案