向网站发送多个请求时,我收到“ 429 Too Many Request”。我希望能够发送多个请求以从URL获得响应(有限制)或批量运行任务,这样我就不会收到来自网站的错误。
下面是我累的代码。它工作正常,但是在发送了少量请求后,却给出了过多的请求消息。
const int maxConcurrency = 15;
try
{
List<Task<HttpResponseMessage>> taskList = new List<Task<HttpResponseMessage>>();
List<string> replies = new List<string>();
for (int i = 0; i < lbURLToSearch.Items.Count; i++)
{
taskList.Add(reusableHTTPClient.GetAsync(lbURLToSearch.Items[i].ToString()));
if (taskList.Count >= maxConcurrency)
Task.WaitAny(taskList.ToArray());
}
var httpClientResponse = await Task.WhenAll(taskList);
foreach (var htmlContent in httpClientResponse)
{
using (var reader = new StreamReader(await htmlContent.Content.ReadAsStreamAsync()))
{
//replies.Add(JsonConvert.DeserializeObject<string>(reader.ReadToEnd()));
rawHTMLFromSource += await reader.ReadToEndAsync();
}
}
foreach (var reply in replies)
{
// TODO:
}
}
catch
{
}
如何减少请求数量?