Question

我试图通过使用带代理的webclient DownloadString方法来请求网页。

以下是我的代码：

using (WebClient wc = new WebClient())
{
    try
    {
        wc.Encoding = Encoding.UTF8;
        wc.Proxy = new WebProxy(myProxy);
        string result = wc.DownloadString(url);
    }
    catch (Exception ex)
    {
        //log exception
    }
}

我有很少的代理，几乎所有代理都可以使用上面的代码。但是，有时响应返回时间非常长（超过一小时），我相信这是由于我的代理缓慢问题。但我不明白为什么它没有抛出异常，因为webclient类应该有一个默认超时（我搜索默认超时应该是100秒）。

因此我想问一下如何避免这种情况？谢谢大家。

Answer 1

因为你需要两个线程 - 一个用于下载，一个用于超时取消。

，这将很复杂

wc.Encoding = Encoding.UTF8;
wc.Proxy = new WebProxy(myProxy);

string result = null;
var waitCancel = new CancellationTokenSource();

wc.DownloadStringCompleted += (sender, e) =>
{
    if (e.Cancelled) return;

    waitCancel.Cancel();

    result = e.Result;
};

wc.DownloadStringAsync(url);

waitCancel.Token.WaitHandle.WaitOne(30 * 1000);
if (waitCancel.IsCancellationRequested == false)
{
    wc.CancelAsync();
}

Console.WriteLine("Result: " + result);

Answer 2

WebClient没有Timeout。你必须像这样扩展它：

using System;
using System.Net;

namespace ConsoleApplication1
{
  internal class Program
  {
    private static void Main(string[] args)
    {
      using (var webClient = new WebClientTimeOut())
      {
        webClient.Encoding = Encoding.UTF8;
        webClient.Proxy = new WebProxy(myProxy);
        try
        {
          var response = webClient.DownloadString("http://www.go1ogle.com");
          Console.WriteLine(response);
        }
        catch (Exception ex)
        {
          Console.WriteLine(ex.ToString());
        }
      }
    }
  }

  public class WebClientTimeOut : WebClient
  {
    protected override WebRequest GetWebRequest(Uri uri)
    {
      var w = base.GetWebRequest(uri);
      w.Timeout = 1000;  // Timeout 1 second
      return w;
    }
  }
}

Answer 3

首先，您需要使用＆＃34; new＆＃34; HttpClient for .Net，下面我将分两步说明

HttpClient具有处理程序，您可以将其置于链中，然后控制整个客户端的行为方式，因此在这种情况下，我们将添加一个进度处理程序，用于指示与传输速度有关的行为

如果传输速度太慢，请取消它并在大多数情况下使用不同的代理启动一个新的...

创建进度处理程序（告诉我们进度如何......如果太慢则抛出异常）

输入：

ProgressMessageHandler ＆＃34; https://msdn.microsoft.com/en-us/library/system.net.http.handlers.progressmessagehandler(v=vs.118).aspx＆＃34;

注意：需要包（System.Net.Http.Formatting.Extension）

我们将用自己的对象包装它以擦除测试并配置为某个预期的传输速率

代码：

    public class MyOwnProgressHandlerContainer
    {
        private readonly long _expectedBytesTransferred;
        private long _lastRecoredBytesTransferred = 0;

        public MyOwnProgressHandlerContainer(long expectedBytesTransferred)
        {
            _expectedBytesTransferred = expectedBytesTransferred;
        }

        public void ReceivedProgressHandler(object sender, HttpProgressEventArgs e)
        {
            // you can uses e.ProgressPercentage but this is calculated based on content length
            // http header which is very much ignored nowadays
            if (_lastRecoredBytesTransferred != 0 && e.BytesTransferred < (_lastRecoredBytesTransferred + _expectedBytesTransferred))
                throw new Exception("Too Slow, Abort !");

            _lastRecoredBytesTransferred = e.BytesTransferred;
        }

        public void SendProgressHandler(object sender, HttpProgressEventArgs e)
        {
            // write stuff to handle here when sending data (mainly for post or uploads)
            Console.WriteLine("Sent data ...");
        }
    }

创建HttpClient并注入我们包装的处理程序：）

输入：

的HttpClient https://msdn.microsoft.com/en-us/library/system.net.http.httpclient(v=vs.118).aspx

代码：

    // set expected rate
    int expectedTransferRate = 25000; // AKA 2.5Mbps

    // create own handler instance 
    MyOwnProgressHandlerContainer myHandler = new MyOwnProgressHandlerContainer(expectedTransferRate);

    // create "ProgressMessageHandler"
    ProgressMessageHandler progresshandler = new ProgressMessageHandler();

    // Hookup event (send)
    progresshandler.HttpSendProgress += myHandler.SendProgressHandler; // these are delegates so they can be a part of a stat-full class

    // Hookup event (Receive)
    progresshandler.HttpReceiveProgress += myHandler.ReceivedProgressHandler; // these are delegates so they can be a part of a stat-full class

    // Create proxy handler
    HttpClientHandler httpClientProxyHandler = 
        new HttpClientHandler
        {
            Proxy = new WebProxy("http://localhost:8888", false),
            UseProxy = true
        };

    // Create client from factory with progress and "proxy" in your case  
    using (HttpClient httpClient = HttpClientFactory.Create(progresshandler, httpClientProxyHandler))
    {
        try
        {
            string downloadResult =
                httpClient
                    // get result (progress handlers are notified based on sent / received data)
                    .GetAsync("https://httpbin.org/image/svg") 
                    // could be a stream to read file content
                    .Result.Content.ReadAsStringAsync().Result; 

                Console.WriteLine(downloadResult);
        }
        catch (Exception)
        {
            // inspected if the exception is the same as the on u throw in    MyOwnProgressHandlerContainer
            throw;
        }
    }

如何使用带有代理的.net webclient来避免不合理的长响应时间？

3 个答案: