如何跟踪异步文件上传到azure存储的进度

时间:2014-01-16 23:48:27

标签: c# file-upload azure azure-storage azure-storage-blobs

有没有办法跟踪文件上传进度到Azure存储容器?
我正在尝试创建一个控制台应用程序,用于使用C#将数据上传到azure 我的coode现在看起来像:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Auth;
using Microsoft.WindowsAzure.Storage.Blob;
using System.Configuration;
using System.IO;
using System.Threading;

namespace AdoAzure
{
    class Program
    {
        static void Main(string[] args)
        {
            CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
            ConfigurationManager.ConnectionStrings["StorageConnectionString"].ConnectionString);
            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
            CloudBlobContainer container = blobClient.GetContainerReference("adokontajnerneki");
            container.CreateIfNotExists();
            CloudBlobClient myBlobClient = storageAccount.CreateCloudBlobClient();
            CloudBlockBlob myBlob = container.GetBlockBlobReference("racuni.adt");
            CancellationToken ca = new CancellationToken();
            var ado = myBlob.UploadFromFileAsync(@"c:\bo\racuni.adt", FileMode.Open, ca);
            Console.WriteLine(ado.Status); //Does Not Help Much
            ado.ContinueWith(t =>
            {
                Console.WriteLine("It is over"); //this is working OK
            });
            Console.WriteLine(ado.Status); //Does Not Help Much
            Console.WriteLine("theEnd");
            Console.ReadKey();
        }
    }
}

这段代码运行良好,但我希望有一些进度条,所以用户可以看到有任务在做。在WindowsAzure.Storage.Blob命名空间中是否有内置的东西,所以我可以用帽子作为兔子?

5 个答案:

答案 0 :(得分:15)

我不认为这是可能的,因为上传文件是一项任务,即使内部文件被拆分为多个块并且这些块被上传,代码实际上等待整个任务完成。

一种可能性是手动将文件拆分为块并使用PutBlockAsync方法异步上传这些块。上传完所有块后,您可以调用PutBlockListAsync方法提交blob。请参阅下面的代码来完成:

using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Auth;
using Microsoft.WindowsAzure.Storage.Blob;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;

namespace ConsoleApplication1
{
    class Program
    {
        static CloudStorageAccount storageAccount = new CloudStorageAccount(new StorageCredentials("accountname", "accountkey"), true);
        static void Main(string[] args)
        {
            CloudBlobClient myBlobClient = storageAccount.CreateCloudBlobClient();
            myBlobClient.SingleBlobUploadThresholdInBytes = 1024 * 1024;
            CloudBlobContainer container = myBlobClient.GetContainerReference("adokontajnerneki");
            //container.CreateIfNotExists();
            CloudBlockBlob myBlob = container.GetBlockBlobReference("cfx.zip");
            var blockSize = 256 * 1024;
            myBlob.StreamWriteSizeInBytes = blockSize;
            var fileName = @"D:\cfx.zip";
            long bytesToUpload = (new FileInfo(fileName)).Length;
            long fileSize = bytesToUpload;

            if (bytesToUpload < blockSize)
            {
                CancellationToken ca = new CancellationToken();
                var ado = myBlob.UploadFromFileAsync(fileName, FileMode.Open, ca);
                Console.WriteLine(ado.Status); //Does Not Help Much
                ado.ContinueWith(t =>
                {
                    Console.WriteLine("Status = " + t.Status);
                    Console.WriteLine("It is over"); //this is working OK
                });
            }
            else
            {
                List<string> blockIds = new List<string>();
                int index = 1;
                long startPosition = 0;
                long bytesUploaded = 0;
                do
                {
                    var bytesToRead = Math.Min(blockSize, bytesToUpload);
                    var blobContents = new byte[bytesToRead];
                    using (FileStream fs = new FileStream(fileName, FileMode.Open))
                    {
                        fs.Position = startPosition;
                        fs.Read(blobContents, 0, (int)bytesToRead);
                    }
                    ManualResetEvent mre = new ManualResetEvent(false);
                    var blockId = Convert.ToBase64String(Encoding.UTF8.GetBytes(index.ToString("d6")));
                    Console.WriteLine("Now uploading block # " + index.ToString("d6"));
                    blockIds.Add(blockId);
                    var ado = myBlob.PutBlockAsync(blockId, new MemoryStream(blobContents), null);
                    ado.ContinueWith(t =>
                    {
                        bytesUploaded += bytesToRead;
                        bytesToUpload -= bytesToRead;
                        startPosition += bytesToRead;
                        index++;
                        double percentComplete = (double)bytesUploaded / (double)fileSize;
                        Console.WriteLine("Percent complete = " + percentComplete.ToString("P"));
                        mre.Set();
                    });
                    mre.WaitOne();
                }
                while (bytesToUpload > 0);
                Console.WriteLine("Now committing block list");
                var pbl = myBlob.PutBlockListAsync(blockIds);
                pbl.ContinueWith(t =>
                {
                    Console.WriteLine("Blob uploaded completely.");
                });
            }
            Console.ReadKey();
        }
    }
}

答案 1 :(得分:4)

Gaurav的解决方案效果很好,与http://blogs.msdn.com/b/kwill/archive/2011/05/30/asynchronous-parallel-block-blob-transfers-with-progress-change-notification.aspx非常相似。这段代码面临的挑战是你在很少的错误处理方面做了很多复杂的工作。我并不是说Gaurav的代码有任何问题 - 它看起来很稳定 - 但特别是对于网络相关的通信代码,你需要考虑很多变量和很多问题。

出于这个原因,我修改了原始博客以使用存储客户端库中的上传代码(假设来自Azure存储团队的代码比我能编写的代码更强大)并使用ProgressStream类跟踪进度。您可以在http://blogs.msdn.com/b/kwill/archive/2013/03/06/asynchronous-parallel-block-blob-transfers-with-progress-change-notification-2-0.aspx找到更新后的代码。

答案 2 :(得分:1)

以下代码使用Azure Blob存储SDK v12。 参考链接:https://www.craftedforeveryone.com/upload-or-download-file-from-azure-blob-storage-with-progress-percentage-csharp/

public void UploadBlob(string fileToUploadPath)
{
    var file = new FileInfo(fileToUploadPath);
    uploadFileSize = file.Length; //Get the file size. This is need to calculate the file upload progress

    //Initialize a progress handler. When the file is being uploaded, the current uploaded bytes will be published back to us using this progress handler by the Blob Storage Service
    var progressHandler = new Progress();
    progressHandler.ProgressChanged += UploadProgressChanged;

    var blob = new BlobClient(connectionString, containerName, file.Name); //Initialize the blob client
    blob.Upload(fileToUploadPath, progressHandler: progressHandler); //Make sure to pass the progress handler here

}

private void UploadProgressChanged(object sender, long bytesUploaded)
{
    //Calculate the progress and update the progress bar.
    //Note: the bytes uploaded published back to us is in long. In order to calculate the percentage, the value has to be converted to double. 
    //Auto type casting from long to double happens here as part of function call
    Console.WriteLine(GetProgressPercentage(uploadFileSize, bytesUploaded));
}

private double GetProgressPercentage(double totalSize,double currentSize)
{
    return (currentSize / totalSize) * 100;
}

答案 3 :(得分:0)

如何?

public class ObservableFileStream : FileStream
{
    private Action<long> _callback;

    public ObservableFileStream(String fileName, FileMode mode, Action<long> callback) : base(fileName, mode)
    {
        _callback = callback;
    }

    public override void Write(byte[] array, int offset, int count)
    {
        _callback?.Invoke(Length);
        base.Write(array, offset, count);
    }

    public override int Read(byte[] array, int offset, int count)
    {
        _callback?.Invoke(Position);
        return base.Read(array, offset, count);
    }
}
public class Test
{
    private async void Upload(String filePath, CloudBlockBlob blob)
    {
        ObservableFileStream fs = null;

        using (fs = new ObservableFileStream(filePath, FileMode.Open, (current) =>
        {
            Console.WriteLine("Uploading " + ((double)current / (double)fs.Length) * 100d);
        }))
        {
            await blob.UploadFromStreamAsync(fs);
        }
    }
}

答案 4 :(得分:0)

使用 BlockBlobClient 而不是 BlobClient (using Azure.Storage.Blobs.Specialized;)

private const string StorageConnectionString = "<Your storage connection string>";
private const string StorageContainerName = "<Your storage container name>";

[HttpPost]
public async Task<IActionResult> UploadFiles(IList<IFormFile> files)
{
    foreach (var file in files)
    {
        var blockBlobClient = new BlockBlobClient(StorageConnectionString, StorageContainerName, file.FileName); // Add Guid to FileName to ensure uniqueness

        using var output = blockBlobClient.OpenWrite(overwrite: true);
        using var input = file.OpenReadStream();

        var buffer = new byte[64 * 1024];
        var totalReadBytes = 0L;
        var readBytes = 0;

        while ((readBytes = input.Read(buffer, 0, buffer.Length)) > 0)
        {
            await output.WriteAsync(buffer, 0, readBytes);

            totalReadBytes += readBytes;

            //progress = Math.Round((totalReadBytes * 1d / sessionUploadFileInfo.FileLength * 100d), 2, MidpointRounding.AwayFromZero);
            //do what you do with the progress, save it in session or however you display it to the user...
            //With WebAPI, I use a static dictionary and a client Id to store file upload progress
            //A separate REST request with client Id gets the file progress' from the dictionary
        }
    }
    return Content("success");
}