Question

我想我已经设法做了一个可重复显示这个问题的测试，至少在我的系统上是这样。 此问题涉及用于错误端点的HttpClient（非远程端点，目标已关闭）。

问题在于完成任务的数量不足总数，通常只有几个。我不介意请求不起作用，但这只会导致应用程序在等待结果时挂在那里。

我从以下测试代码中得到以下结果：

经历：237.2009884秒。批处理数组中的任务：8000个已完成的任务：7993

如果我将batchsize设置为8而不是8000，则完成。对于8000，它会在WhenAll上进行阻塞。

我想知道其他人是否会得到相同的结果，如果我做错了什么，如果这似乎是一个错误。

using System;
using System.Diagnostics;
using System.Linq;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;

namespace CustomArrayTesting
{

    /// <summary>
    /// Problem: a large batch of async http requests is done in a loop using HttpClient, and a few of them never complete
    /// </summary>
    class ProgramTestHttpClient
    {
        static readonly int batchSize = 8000; //large batch size brings about the problem

        static readonly Uri Target = new Uri("http://localhost:8080/BadAddress");

        static TimeSpan httpClientTimeout = TimeSpan.FromSeconds(3);  // short Timeout seems to bring about the problem.

        /// <summary>
        /// Sends off a bunch of async httpRequests using a loop, and then waits for the batch of requests to finish.
        /// I installed asp.net web api client libraries Nuget package.
        /// </summary>
        static void Main(String[] args)
        {
            httpClient.Timeout = httpClientTimeout; 

            stopWatch = new Stopwatch();
            stopWatch.Start();


            // this timer updates the screen with the number of completed tasks in the batch (See timerAction method bellow Main)
            TimerCallback _timerAction = timerAction;
            TimerCallback _resetTimer = ResetTimer;
            TimerCallback _timerCallback = _timerAction + _resetTimer;

            timer = new Timer(_timerCallback, null, TimeSpan.FromSeconds(1), Timeout.InfiniteTimeSpan);
            //

            for (int i = 0; i < batchSize; i++)
            {
                Task<HttpResponseMessage> _response = httpClient.PostAsJsonAsync<Object>(Target, new Object());//WatchRequestBody()

                Batch[i] = _response;
            }

            try
            {
                Task.WhenAll(Batch).Wait();
            }
            catch (Exception ex)
            {

            }

            timer.Dispose();
            timerAction(null);
            stopWatch.Stop();


            Console.WriteLine("Done");
            Console.ReadLine();
        }

        static readonly TimeSpan timerRepeat = TimeSpan.FromSeconds(1);

        static readonly HttpClient httpClient = new HttpClient();

        static Stopwatch stopWatch;

        static System.Threading.Timer timer;

        static readonly Task[] Batch = new Task[batchSize];

        static void timerAction(Object state)
        {
            Console.Clear();
            Console.WriteLine("Elapsed: {0} seconds.", stopWatch.Elapsed.TotalSeconds);
            var _tasks = from _task in Batch where _task != null select _task;
            int _tasksCount = _tasks.Count();

            var _completedTasks = from __task in _tasks where __task.IsCompleted select __task;
            int _completedTasksCount = _completedTasks.Count();

            Console.WriteLine("Tasks in batch array: {0}       Completed Tasks : {1} ", _tasksCount, _completedTasksCount);

        }

        static void ResetTimer(Object state)
        {
            timer.Change(timerRepeat, Timeout.InfiniteTimeSpan);
        }
    }
}

有时它会在完成Access Violation未处理的异常之前崩溃。调用堆栈只是说：

>   mscorlib.dll!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(uint errorCode = 1225, uint numBytes = 0, System.Threading.NativeOverlapped* pOVERLAP = 0x08b38b98) 
    [Native to Managed Transition]  
    kernel32.dll!@BaseThreadInitThunk@12()  
    ntdll.dll!___RtlUserThreadStart@8()     
    ntdll.dll!__RtlUserThreadStart@8()

大部分时间它都没有崩溃但只是永远不会等待时间。在任何情况下，每个请求都会抛出以下第一次机会异常：

A first chance exception of type 'System.Net.Sockets.SocketException' occurred in System.dll
A first chance exception of type 'System.Net.WebException' occurred in System.dll
A first chance exception of type 'System.AggregateException' occurred in mscorlib.dll
A first chance exception of type 'System.ObjectDisposedException' occurred in System.dll

我使调试器停止在Object dispose异常上，并获得了这个调用堆栈：

>   System.dll!System.Net.Sockets.NetworkStream.UnsafeBeginWrite(byte[] buffer, int offset, int size, System.AsyncCallback callback, object state) + 0x136 bytes    
    System.dll!System.Net.PooledStream.UnsafeBeginWrite(byte[] buffer, int offset, int size, System.AsyncCallback callback, object state) + 0x19 bytes  
    System.dll!System.Net.ConnectStream.WriteHeaders(bool async = true) + 0x105 bytes   
    System.dll!System.Net.HttpWebRequest.EndSubmitRequest() + 0x8a bytes    
    System.dll!System.Net.HttpWebRequest.SetRequestSubmitDone(System.Net.ConnectStream submitStream) + 0x11d bytes  
    System.dll!System.Net.Connection.CompleteConnection(bool async, System.Net.HttpWebRequest request = {System.Net.HttpWebRequest}) + 0x16c bytes  
    System.dll!System.Net.Connection.CompleteConnectionWrapper(object request, object state) + 0x4e bytes   
    System.dll!System.Net.PooledStream.ConnectionCallback(object owningObject, System.Exception e, System.Net.Sockets.Socket socket, System.Net.IPAddress address) + 0xf0 bytes 
    System.dll!System.Net.ServicePoint.ConnectSocketCallback(System.IAsyncResult asyncResult) + 0xe6 bytes  
    System.dll!System.Net.LazyAsyncResult.Complete(System.IntPtr userToken) + 0x65 bytes    
    System.dll!System.Net.ContextAwareResult.Complete(System.IntPtr userToken) + 0x92 bytes 
    System.dll!System.Net.LazyAsyncResult.ProtectedInvokeCallback(object result, System.IntPtr userToken) + 0xa6 bytes  
    System.dll!System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* nativeOverlapped) + 0x98 bytes 
    mscorlib.dll!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* pOVERLAP) + 0x6e bytes    
    [Native to Managed Transition]

异常消息是：

{"Cannot access a disposed object.\r\nObject name: 'System.Net.Sockets.NetworkStream'."}    System.Exception {System.ObjectDisposedException}

请注意我很少看到的与未处理的访问冲突异常的关系。

因此，似乎HttpClient在目标停机时不健壮。顺便说一下，我在Windows 7 32上这样做。

Answer 1

我使用反射器查看了HttpClient的来源。对于同步执行的操作部分（当它被启动时），就我所见，似乎没有应用于返回任务的超时。有一些超时实现在HttpWebRequest对象上调用Abort（），但它们似乎又错过了异步函数这一侧返回任务的任何超时取消或故障。回调方面可能还有一些东西，但有时回调可能会丢失＆＃34;导致返回的任务永远不会完成。

我发布了一个问题，询问如何为任何任务添加超时，并且回答者提供了这个非常好的解决方案（这里作为扩展方法）：

public static Task<T> WithTimeout<T>(this Task<T> task, TimeSpan timeout)
{
    var delay = task.ContinueWith(t => t.Result
        , new CancellationTokenSource(timeout).Token);
    return Task.WhenAny(task, delay).Unwrap();
}

因此，像这样调用HttpClient可以防止任何＆＃34;任务变坏＆＃34;永无止境：

Task<HttpResponseMessage> _response = httpClient.PostAsJsonAsync<Object>(Target, new Object()).WithTimeout<HttpResponseMessage>(httpClient.Timeout);

我认为还有一些事情要求不太可能丢失： 1.将超时从3秒增加到30秒，使得我在这个问题上发布的程序中完成了所有任务。 2.增加允许使用的并发连接数，例如System.Net.ServicePointManager.DefaultConnectionLimit = 100;

Answer 2

我在Google搜索来自WCF的类似问题的解决方案时遇到了这个问题。这一系列例外与我看到的模式完全相同。最后通过大量调查我发现了HttpClient使用的HttpWebRequest中的一个错误。 HttpWebRequest处于错误状态，只发送HTTP头。然后它坐着等待永远不会发送的响应。

我已经使用Microsoft Connect提出了一张票，可以在此处找到：https://connect.microsoft.com/VisualStudio/feedback/details/1805955/async-post-httpwebrequest-hangs-when-a-socketexception-occurs-during-setsocketoption

细节在票证中，但它需要从HttpWebRequest到非localhost机器的异步POST调用。我在.Net 4.5和4.6的Windows 7上重现了它。引发SocketException的失败的SetSocketOption调用仅在测试时在Windows 7上失败。

对于我们来说，UseNagleAlgorithm设置会导致SetSocketOption调用，但是我们无法避免它，因为WCF关闭了UseNagleAlgorithm并且您无法阻止它。在WCF中，它显示为超时调用。显然，这并不是很好，因为我们花了60多岁时一无所获。

Answer 3

您的异常信息在WhenAll任务中丢失。不要使用它，试试这个：

Task aggregateTask = Task.Factory.ContinueWhenAll(
    Batch,
    TaskExtrasExtensions.PropagateExceptions,
    TaskContinuationOptions.ExecuteSynchronously);

aggregateTask.Wait();

这使用Parallel Extensions Extras示例代码中的PropagateExceptions扩展方法来确保批处理操作中的任务中的异常信息不会丢失：

/// <summary>Propagates any exceptions that occurred on the specified tasks.</summary>
/// <param name="tasks">The Task instances whose exceptions are to be propagated.</param>
public static void PropagateExceptions(this Task [] tasks)
{
    if (tasks == null) throw new ArgumentNullException("tasks");
    if (tasks.Any(t => t == null)) throw new ArgumentException("tasks");
    if (tasks.Any(t => !t.IsCompleted)) throw new InvalidOperationException("A task has not completed.");
    Task.WaitAll(tasks);
}

HttpClient异步请求未完成在循环中发送的大批量

3 个答案: