重用HttpClient进行多部分下载

时间:2018-05-03 01:54:07

标签: c# task httpclient download-manager

我正在重复HttpClient用于我的所有下载任务,因为我总是在SO中看到我应该重复使用httpclient。但是我收到此异常Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The request was aborted: The request was canceled.

这是我到目前为止所做的事情。
1.为每个下载任务创建HttpClient的新实例(工作,但不重用httpclient)
2.将任务数量减少到2个(工作,2个以上的任何事件抛出异常)
3.从请求消息中删除HttpCompletionOption.ResponseHeadersRead(有效,但启动下载需要很长时间,我认为它会在启动流之前继续读取所有文件)
4.使HttpClient超时无限(没有工作)
5.使用HttpContent.ReadAsStreamAsync方法并将读取超时设置为无限(没有工作)

以下是代码

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;

namespace ConsoleApp3
{
    class Program
    {
        static void Main(string[] args)
        {
            Task.Run(() => new Downloader().Download(
                "https://ia800703.us.archive.org/34/items/1mbFile/1mb.mp4",
                "1mb.mp4"
            )).Wait();
        }
    }

    public class Downloader
    {
        HttpClient httpClient;

        public Downloader()
        {
            httpClient = new HttpClient();
        }

        public async Task Download(string url, string saveAs)
        {
            var response = await httpClient.SendAsync(new HttpRequestMessage(HttpMethod.Head, url), HttpCompletionOption.ResponseHeadersRead);
            var parallelDownloadSuported = response.Headers.AcceptRanges.Contains("bytes");
            var contentLength = response.Content.Headers.ContentLength ?? 0;

            if (parallelDownloadSuported)
            {
                const double numberOfParts = 5.0;
                var tasks = new List<Task>();
                var partSize = (long)Math.Ceiling(contentLength / numberOfParts);

                File.Create(saveAs).Dispose();

                for (var i = 0; i < numberOfParts; i++)
                {
                    var start = i * partSize + Math.Min(1, i);
                    var end = Math.Min((i + 1) * partSize, contentLength);

                    tasks.Add(
                        Task.Run(() => DownloadPart(url, saveAs, start, end))
                        );
                }

                await Task.WhenAll(tasks);
            }
        }

        private async void DownloadPart(string url, string saveAs, long start, long end)
        {
            using (var fileStream = new FileStream(saveAs, FileMode.Open, FileAccess.Write, FileShare.Write, 1024, true))
            {
                var message = new HttpRequestMessage(HttpMethod.Get, url);
                message.Headers.Add("Range", string.Format("bytes={0}-{1}", start, end));

                fileStream.Position = start;
                await httpClient.SendAsync(message, HttpCompletionOption.ResponseHeadersRead).Result.Content.CopyToAsync(fileStream);
            }
        }
    }
}

代码来自:here

0 个答案:

没有答案