Question

我在Windows服务中从网上下载了100多个大文件 - c＃。要求是一次维护 - 最多4个并行Web文件下载。

我可以使用async await实现并发/并行下载，还是必须使用BackgroundWorker进程或线程？ async-await多线程？请参阅下面的async-await查看我的示例程序：

    static int i = 0;

    Timer_tick()
    {
      while(i < 4)
      {
        i++;
        model = GetNextModel();
        await Download(model);
      }
    }

    private async Download(XYZ model)
    {
    Task<FilesetResult> t = DoWork(model);
    result = await t; 
    //Use Result
    }

    private async Task<FilesetResult> Work(XYZ model)
    {
    fileresult = await api.Download(model.path)
    i--;
    return filesetresult;
    }

Answer 1

您可以使用SemaphoreSlim类限制并行运行的异步任务的数量。类似的东西：

List<DownloadRequest> requests = Enumerable.Range(0, 100).Select(x => new DownloadRequest()).ToList();
using (var throttler = new SemaphoreSlim(4))
{
    Task<DownloadResult>[] downloadTasks = requests.Select(request => Task.Run(async () =>
    {
        await throttler.WaitAsync();
        try
        {
            return await DownloadTaskAsync(request);
        }
        finally
        {
            throttler.Release();
        }
    })).ToArray();
    await Task.WhenAll(downloadTasks);
}

更新：感谢您提出意见，修复了问题。

Update2 ：动态请求列表的示例解决方案

public class DownloadManager : IDisposable
{
    private readonly SemaphoreSlim _throttler = new SemaphoreSlim(4);

    public async Task<DownloadResult> DownloadAsync(DownloadRequest request)
    {
        await _throttler.WaitAsync();
        try
        {
            return await api.Download(request);
        }
        finally
        {
            _throttler.Release();
        }
    }

    public void Dispose()
    {
        _throttler?.Dispose();
    }
}

Answer 2

手工操作似乎非常复杂。

var files = new List<Uri>();

Parallel.ForEach(files, 
                 new ParallelOptions { MaxDegreeOfParallelism = 4 },
                 this.Download);

现在你只需要一个正常的同步方法private void Download(Uri file)，你就可以了。

如果您需要生产者/消费者模式，最简单的版本可能是BlockingCollection：

using System;
using System.Collections.Concurrent;
using System.Threading;
using System.Threading.Tasks;

namespace ConsoleApp11
{
    internal class Program
    {
        internal static void Main()
        {
            using (var queue = new BlockingCollection<Uri>())
            {
                // starting the producer task:
                Task.Factory.StartNew(() =>
                {
                    for (int i = 0; i < 100; i++)
                    {
                        // faking read from message queue... we get a new Uri every 100 ms
                        queue.Add(new Uri("http://www.example.com/" + i));

                        Thread.Sleep(100);
                    }

                    // just to end this program... you don't need to end this, just listen to your message queue
                    queue.CompleteAdding();
                });

                // run the consumers:
                Parallel.ForEach(queue.GetConsumingEnumerable(), new ParallelOptions { MaxDegreeOfParallelism = 4 }, Download);
            }
        }

        internal static void Download(Uri uri)
        {
            // download your file here

            Console.WriteLine($"Downloading {uri} [..        ]");
            Thread.Sleep(1000);
            Console.WriteLine($"Downloading {uri} [.....     ]");
            Thread.Sleep(1000);
            Console.WriteLine($"Downloading {uri} [.......   ]");
            Thread.Sleep(1000);
            Console.WriteLine($"Downloading {uri} [......... ]");
            Thread.Sleep(1000);
            Console.WriteLine($"Downloading {uri} [..........]");
            Thread.Sleep(1000);
            Console.WriteLine($"Downloading {uri} OK");
        }
    }
}

使用async-await进行并行多线程下载

2 个答案: