Question

说明

我想使用WebClient的DownloadStringAsync下载多个URL（大约300个），一些URL下载得很好，但对于其他一些URL下载只是永久挂起。

问题：

如何捕获无法下载的有问题的网址？
根据附带的代码，我如何实现“DownloadStringAsync”方法的超时期限，以跳过无法下载的网址？

我已经google了解决方案但是缺乏信心。然后尝试在此处搜索数据库中的解决方案，结果相同。任何帮助表示赞赏。

这是我正在使用的代码：

public class ParallelDownloading
    {
        private int _totalDownloadedURLs = 0;
        private ConcurrentQueue<DownloadFile> _queueToDownlaod;
        private IList<Task> _downloadingTasks;
        private Timer _downloadTimer;

        public event EventHandler<OnFileDownloadCompletedEventArgs> OnDownloadURLCompleted;

        private int _parallelDownloads;

        public ParallelDownloading(int parallelDownloads)
        {
            _totalDownloadedURLs = 0;
            _queueToDownlaod = new ConcurrentQueue<DownloadFile>();
            _downloadingTasks = new List<Task>();
            _downloadTimer = new Timer();

            _parallelDownloads = parallelDownloads;

            ServicePointManager.DefaultConnectionLimit = parallelDownloads;
        }

        public void StartParallelDownloading()
        {
            _downloadTimer.Elapsed += new ElapsedEventHandler(DownloadTimer_Elapsed);
            _downloadTimer.Interval = 800;
            _downloadTimer.Start();
        }

        public void EnqueueFileToDownload(DownloadFile file)
        {
            _queueToDownlaod.Enqueue(file);
        }

        void DownloadTimer_Elapsed(object sender, ElapsedEventArgs e)
        {
            StartDownload();
        }

        private void StartDownload()
        {
            lock (_downloadingTasks)
            {
                if (_downloadingTasks.Count < _parallelDownloads && _queueToDownlaod.Count > 0)
                {
                    DownloadFile fileToDownload;
                    if (_queueToDownlaod.TryDequeue(out fileToDownload))
                    {
                        var task = new Task(() =>
                        {
                            var client = new WebClient();
                            client.DownloadStringCompleted += new DownloadStringCompletedEventHandler(client_DownloadStringCompleted);
                            client.DownloadStringAsync(new Uri(fileToDownload.Link), fileToDownload.Link);

                        }, TaskCreationOptions.LongRunning);

                        task.ContinueWith(DownloadOverCallback, TaskContinuationOptions.None);

                        _downloadingTasks.Add(task);
                        task.Start();
                    }
                }
            }
        }

        void client_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
        {
            _totalDownloadedURLs ++;
            if (e.Error == null && e.Cancelled == false)
            {
                //do something
            }
            OnFileDownloadCompletedEventArgs evtArgs = new OnFileDownloadCompletedEventArgs();
            evtArgs.DownloadedFiles = _totalDownloadedURLs;
            OnDownloadURLCompleted(this, evtArgs);
        }

        public void DownloadOverCallback(Task downloadingTask)
        {
            lock (_downloadingTasks)
            {
                _downloadingTasks.Remove(downloadingTask);
            }
        }
    }

Answer 1

1.如何捕获无法下载的有问题的网址？为此你可以使用try，catch来看看那里发生了什么

2.基于附带的代码，我如何实现'DownloadStringAsync'方法的超时期限，以跳过无法下载的Url？最简单的方法是使用Timer在一段时间后调用CancelAsync或使用具有Timeout属性的HttpWebRequest

DownloadStringAsync并行300个URL - C＃

1 个答案: