Question

使用以下代码：

static void Main(string[] args)
{
    ServicePointManager.DefaultConnectionLimit = 1000;

    var client = new HttpClient();
    var uris = File.ReadAllLines(@"C:\urls.txt").Select(x => new Uri(x));

    foreach(var uri in uris)
    {
        var url = uri.ToString();

        var task = client.GetStringAsync(uri);
        task.ContinueWith(t => Console.WriteLine("Done {0}", url), TaskContinuationOptions.OnlyOnRanToCompletion);
        task.ContinueWith(t => Console.WriteLine("Failed {0}", url), TaskContinuationOptions.OnlyOnFaulted);
        task.ContinueWith(t => Console.WriteLine("Cancelled {0}", url), TaskContinuationOptions.OnlyOnCanceled);
    }

    Console.ReadKey();
}

根据提琴手的说法，我最多可以同时要求15-20个网址。所有这些网址都是唯一的，并没有指向同一个主机。

发生了什么事？

Answer 1

运行此功能的计算机上的CPU有多少个核心？机器可以处理的并发操作数量有限制。此外，TPL会根据手头的任务自动决定调用的正确并行数量。旋转1000个线程来完成任务并不总是更有效。在线程之间传递了大量的无意中管理消息。

这可能没有任何性能改进，但对于并行性来说这应该更为惯用：

static void Main(string[] args)
{
    ServicePointManager.DefaultConnectionLimit = 1000;

    var uris = File.ReadAllLines(@"C:\urls.txt").Select(x => new Uri(x));

    foreach(var uri in uris)
    {
        var client = new HttpClient();
        var url = uri.ToString();

        var task = client.GetStringAsync(uri);
        task.ContinueWith(t => Console.WriteLine("Done {0}", url), TaskContinuationOptions.OnlyOnRanToCompletion);
        task.ContinueWith(t => Console.WriteLine("Failed {0}", url), TaskContinuationOptions.OnlyOnFaulted);
        task.ContinueWith(t => Console.WriteLine("Cancelled {0}", url), TaskContinuationOptions.OnlyOnCanceled);
    }

    Console.ReadKey();
}

甚至：

static void Main(string[] args)
{
    ServicePointManager.DefaultConnectionLimit = 1000;

    var uris = File.ReadAllLines(@"C:\urls.txt").Select(x => new Uri(x));

    Parallel.ForEach(uris, uri => {
        WebRequest myRequest = WebRequest.Create(uri.toString());
        // handle response synchronously
    });

}

Web请求数量有限

1 个答案: