Web API - 上传到Azure存储时取得进展

时间:2015-09-21 13:47:47

标签: c# azure asp.net-web-api2 azure-storage-blobs

我想要完成的任务是创建一个Web API服务,以便将文件上传到Azure存储。同时,我希望有一个反映实际上传进度的进度指示器。经过一些研究和研究,我发现了两件重要的事情:

首先,我必须手动将文件拆分为块,然后使用 Microsoft.WindowsAzure.Storage.dll 中的PutBlockAsync方法异步上传它们。

其次,我必须在流媒体模式下的Web API服务中接收文件,而不是在缓冲模式下。

所以到目前为止,我有以下实现:

UploadController.cs

using System.Configuration;
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;
using System.Web.Http;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using WebApiFileUploadToAzureStorage.Infrastructure;
using WebApiFileUploadToAzureStorage.Models;

namespace WebApiFileUploadToAzureStorage.Controllers
{
    public class UploadController : ApiController
    {
        [HttpPost]
        public async Task<HttpResponseMessage> UploadFile()
        {
            if (!Request.Content.IsMimeMultipartContent("form-data"))
            {
                return Request.CreateResponse(HttpStatusCode.UnsupportedMediaType,
                    new UploadStatus(null, false, "No form data found on request.", string.Empty, string.Empty));
            }

            var streamProvider = new MultipartAzureBlobStorageProvider(GetAzureStorageContainer());
            var result = await Request.Content.ReadAsMultipartAsync(streamProvider);

            if (result.FileData.Count < 1)
            {
                return Request.CreateResponse(HttpStatusCode.BadRequest,
                    new UploadStatus(null, false, "No files were uploaded.", string.Empty, string.Empty));
            }

            return Request.CreateResponse(HttpStatusCode.OK);
        }

        private static CloudBlobContainer GetAzureStorageContainer()
        {
            var storageConnectionString = ConfigurationManager.AppSettings["AzureBlobStorageConnectionString"];
            var storageAccount = CloudStorageAccount.Parse(storageConnectionString);

            var blobClient = storageAccount.CreateCloudBlobClient();
            blobClient.DefaultRequestOptions.SingleBlobUploadThresholdInBytes = 1024 * 1024;

            var container = blobClient.GetContainerReference("photos");

            if (container.Exists())
            {
                return container;
            }

            container.Create();

            container.SetPermissions(new BlobContainerPermissions
            {
                PublicAccess = BlobContainerPublicAccessType.Container
            });

            return container;
        }
    }
}

MultipartAzureBlobStorageProvider.cs

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Net.Http;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.WindowsAzure.Storage.Blob;

namespace WebApiFileUploadToAzureStorage.Infrastructure
{
    public class MultipartAzureBlobStorageProvider : MultipartFormDataStreamProvider
    {
        private readonly CloudBlobContainer _blobContainer;

        public MultipartAzureBlobStorageProvider(CloudBlobContainer blobContainer) : base(Path.GetTempPath())
        {
            _blobContainer = blobContainer;
        }

        public override Task ExecutePostProcessingAsync()
        {
            const int blockSize = 256 * 1024;
            var fileData = FileData.First();
            var fileName = Path.GetFileName(fileData.Headers.ContentDisposition.FileName.Trim('"'));
            var blob = _blobContainer.GetBlockBlobReference(fileName);
            var bytesToUpload = (new FileInfo(fileData.LocalFileName)).Length;
            var fileSize = bytesToUpload;

            blob.Properties.ContentType = fileData.Headers.ContentType.MediaType;
            blob.StreamWriteSizeInBytes = blockSize;

            if (bytesToUpload < blockSize)
            {
                var cancellationToken = new CancellationToken();

                using (var fileStream = new FileStream(fileData.LocalFileName, FileMode.Open, FileAccess.ReadWrite))
                {
                    var upload = blob.UploadFromStreamAsync(fileStream, cancellationToken);

                    Debug.WriteLine($"Status {upload.Status}.");

                    upload.ContinueWith(task =>
                    {
                        Debug.WriteLine($"Status {task.Status}.");
                        Debug.WriteLine("Upload is over successfully.");
                    }, TaskContinuationOptions.OnlyOnRanToCompletion);

                    upload.ContinueWith(task =>
                    {
                        Debug.WriteLine($"Status {task.Status}.");

                        if (task.Exception != null)
                        {
                            Debug.WriteLine("Task could not be completed." + task.Exception.InnerException);
                        }
                    }, TaskContinuationOptions.OnlyOnFaulted);

                    upload.Wait(cancellationToken);
                }
            }
            else
            {
                var blockIds = new List<string>();
                var index = 1;
                long startPosition = 0;
                long bytesUploaded = 0;

                do
                {
                    var bytesToRead = Math.Min(blockSize, bytesToUpload);
                    var blobContents = new byte[bytesToRead];

                    using (var fileStream = new FileStream(fileData.LocalFileName, FileMode.Open))
                    {
                        fileStream.Position = startPosition;
                        fileStream.Read(blobContents, 0, (int)bytesToRead);
                    }

                    var manualResetEvent = new ManualResetEvent(false);
                    var blockId = Convert.ToBase64String(Encoding.UTF8.GetBytes(index.ToString("d6")));
                    Debug.WriteLine($"Now uploading block # {index.ToString("d6")}");
                    blockIds.Add(blockId);
                    var upload = blob.PutBlockAsync(blockId, new MemoryStream(blobContents), null);

                    upload.ContinueWith(task =>
                    {
                        bytesUploaded += bytesToRead;
                        bytesToUpload -= bytesToRead;
                        startPosition += bytesToRead;
                        index++;
                        var percentComplete = (double)bytesUploaded / fileSize;
                        Debug.WriteLine($"Percent complete: {percentComplete.ToString("P")}");
                        manualResetEvent.Set();
                    });

                    manualResetEvent.WaitOne();
                } while (bytesToUpload > 0);

                Debug.WriteLine("Now committing block list.");
                var putBlockList = blob.PutBlockListAsync(blockIds);

                putBlockList.ContinueWith(task =>
                {
                    Debug.WriteLine("Blob uploaded completely.");
                });

                putBlockList.Wait();
            }

            File.Delete(fileData.LocalFileName);
            return base.ExecutePostProcessingAsync();
        }
    }
}

我还启用了流媒体模式,this博文建议。就文件成功上载到Azure存储而言,此方法非常有用。然后,当我使用XMLHttpRequest(并订阅进度事件)来调用此服务时,我看到指标很快就会移动到100%。如果5MB文件需要大约1分钟上传,我的指标只需1秒即可移动到最后。因此,问题可能在于服务器通知客户端上载进度的方式。有什么想法吗?谢谢。

================================更新1 ============= ======================

这是我用来调用服务的JavaScript代码

function uploadFile(file, index, uploadCompleted) {
    var authData = localStorageService.get("authorizationData");
    var xhr = new XMLHttpRequest();

    xhr.upload.addEventListener("progress", function (event) {
        fileUploadPercent = Math.floor((event.loaded / event.total) * 100);
        console.log(fileUploadPercent + " %");
    });

    xhr.onreadystatechange = function (event) {
        if (event.target.readyState === event.target.DONE) {

            if (event.target.status !== 200) {
            } else {
                var parsedResponse = JSON.parse(event.target.response);
                uploadCompleted(parsedResponse);
            }

        }
    };

    xhr.open("post", uploadFileServiceUrl, true);
    xhr.setRequestHeader("Authorization", "Bearer " + authData.token);

    var data = new FormData();
    data.append("file-" + index, file);

    xhr.send(data);
}

2 个答案:

答案 0 :(得分:6)

您的进度指示器可能会快速移动,可能是因为

public async Task<HttpResponseMessage> UploadFile()

我之前遇到过这种情况,在创建异步类型的api时,我甚至不确定是否可以等待,它当然只是在后台完成你的api调用,因为你的进度指示器立即完成,因为异步方法(火与忘)。 api会立即给你回复,但实际上会在服务器后台完成(如果没有等待)。

请尝试制作它

public HttpResponseMessage UploadFile()

并尝试这些

var result = Request.Content.ReadAsMultipartAsync(streamProvider).Result;
var upload = blob.UploadFromStreamAsync(fileStream, cancellationToken).Result;

OR

var upload = await blob.UploadFromStreamAsync(fileStream, cancellationToken);
希望它有所帮助。

答案 1 :(得分:2)

其他方式来完成你想要的东西(我不明白XMLHttpRequest的progress事件是如何工作的)正在使用ProgressMessageHandler来获取请求中的上传进度。然后,为了通知客户端,您可以使用一些缓存来存储进度,并从客户端请求其他端点中的当前状态,或使用SignalR将进度从服务器发送到客户端

类似的东西:

//WebApiConfigRegister
var progress = new ProgressMessageHandler();
progress.HttpSendProgress += HttpSendProgress;
config.MessageHandlers.Add(progress);
//End WebApiConfig Register

    private static void HttpSendProgress(object sender, HttpProgressEventArgs e)
    {   
        var request = sender as HttpRequestMessage;
        //todo: check if request is not null
        //Get an Id from the client or something like this to identify the request
        var id = request.RequestUri.Query[0];
        var perc = e.ProgressPercentage;
        var b = e.TotalBytes;
        var bt = e.BytesTransferred;
        Cache.InsertOrUpdate(id, perc);
    }

您可以查看更多文档on this MSDN blog post(向下滚动到“进度通知”部分)

此外,您可以根据数据块计算进度,将进度存储在缓存中,并以与上述相同的方式进行通知。 Something like this solution