批量媒体通过WebApi上载到Azure Blob存储

时间:2017-07-28 15:31:35

标签: javascript c# azure asp.net-web-api azure-storage-blobs

我的网络应用目前允许用户使用以下方法一次上传媒体:

var fd = new FormData(document.forms[0]);
fd.append("media", blob); // blob is the image/video
$.ajax({
    type: "POST",
    url: '/api/media',
    data: fd
})

媒体然后被发布到WebApi控制器:

    [HttpPost, Route("api/media")]
    public async Task<IHttpActionResult> UploadFile()
    {
        if (!Request.Content.IsMimeMultipartContent("form-data"))
        {
            throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
        }

        string mediaPath = await _mediaService.UploadFile(User.Identity.Name, Request.Content);

        return Ok(mediaPath);
    }

然后按照以下方式执行:

 public async Task<string> UploadFile(string username, HttpContent content)
 {
   var storageAccount = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
   CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
   CloudBlobContainer imagesContainer = blobClient.GetContainerReference("container-" + user.UserId);
   var provider = new AzureStorageMultipartFormDataStreamProvider(imagesContainer);
   await content.ReadAsMultipartAsync(provider);
   var filename = provider.FileData.FirstOrDefault()?.LocalFileName;
   // etc
 }

这对于单个上传非常有用,但是如何通过单个流操作返回上传的文件名数组来修改此内容以支持多个文件的批量上传?关于此的文档/示例似乎很少。

public class AzureStorageMultipartFormDataStreamProvider : MultipartFormDataStreamProvider
{
    private readonly CloudBlobContainer _blobContainer;
    private readonly string[] _supportedMimeTypes = { "images/png", "images/jpeg", "images/jpg", "image/png", "image/jpeg", "image/jpg", "video/webm" };

    public AzureStorageMultipartFormDataStreamProvider(CloudBlobContainer blobContainer) : base("azure")
    {
        _blobContainer = blobContainer;
    }

    public override Stream GetStream(HttpContent parent, HttpContentHeaders headers)
    {
        if (parent == null) throw new ArgumentNullException(nameof(parent));
        if (headers == null) throw new ArgumentNullException(nameof(headers));

        if (!_supportedMimeTypes.Contains(headers.ContentType.ToString().ToLower()))
        {
            throw new NotSupportedException("Only jpeg and png are supported");
        }

        // Generate a new filename for every new blob
        var fileName = Guid.NewGuid().ToString();

        CloudBlockBlob blob = _blobContainer.GetBlockBlobReference(fileName);

        if (headers.ContentType != null)
        {
            // Set appropriate content type for your uploaded file
            blob.Properties.ContentType = headers.ContentType.MediaType;
        }

        this.FileData.Add(new MultipartFileData(headers, blob.Name));

        return blob.OpenWrite();
    }
}

3 个答案:

答案 0 :(得分:2)

假设您的AzureStorageMultipartFormDataStreamProviderthis blog中提及的同一类相似,如果请求中有多个文件,则实际上已经处理了多个文件。

所以您需要做的就是更改UploadFile以返回IEnumerable<string>并将控制器更改为mediaPath

所以你的MediaService会有:

var filenames = provider.FileData.Select(x => x.LocalFileName).ToList(); ;
return filenames;

你的控制器会:

var mediaPaths = await _mediaService.UploadFile(User.Identity.Name, Request.Content);
return Ok(mediaPaths);

答案 1 :(得分:1)

由于您未使用AzureStorageMultipartFormDataStreamProvider类发布相关代码。

所以我创建了一个自定义的AzureStorageMultipartFormDataStreamProvider,它继承自MultipartFileStreamProvider,以启用web api上传批量上传的多个文件。

在AzureStorageMultipartFormDataStreamProvider中,我们可以覆盖ExecutePostProcessingAsync方法。

在这种方法中,我们可以获取上传文件数据,然后我们可以将这些数据上传到azure存储。

更多细节,您可以参考以下代码。总控制器。

 public class UploadingController : ApiController
    {
        public Task<List<FileItem>> PostFile()
        {
            if (!Request.Content.IsMimeMultipartContent("form-data"))
            {
                throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
            }

            var multipartStreamProvider = new AzureStorageMultipartFormDataStreamProvider(GetWebApiContainer());
            return Request.Content.ReadAsMultipartAsync<AzureStorageMultipartFormDataStreamProvider>(multipartStreamProvider).ContinueWith<List<FileItem>>(t =>
            {
                if (t.IsFaulted)
                {
                    throw t.Exception;
                }

                AzureStorageMultipartFormDataStreamProvider provider = t.Result;
                return provider.Files;
            });
        }


        public static CloudBlobContainer GetWebApiContainer(string containerName = "webapi-file-container")
        {
            // Retrieve storage account from connection-string
            CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
               "your connection string");

            // Create the blob client 
            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

            CloudBlobContainer container = blobClient.GetContainerReference(containerName);

            // Create the container if it doesn't already exist
            container.CreateIfNotExists();

            // Enable public access to blob
            var permissions = container.GetPermissions();
            if (permissions.PublicAccess == BlobContainerPublicAccessType.Off)
            {
                permissions.PublicAccess = BlobContainerPublicAccessType.Blob;
                container.SetPermissions(permissions);
            }

            return container;
        }
    }


    public class FileItem
    {
        /// <summary>
        /// file name
        /// </summary>
        public string Name { get; set; }
        /// <summary>
        /// size in bytes
        /// </summary>
        public string SizeInMB { get; set; }
        public string ContentType { get; set; }
        public string Path { get; set; }
        public string BlobUploadCostInSeconds { get; set; }
    }


    public class AzureStorageMultipartFormDataStreamProvider : MultipartFileStreamProvider
    {
        private CloudBlobContainer _container;
        public AzureStorageMultipartFormDataStreamProvider(CloudBlobContainer container)
            : base(Path.GetTempPath())
        {
            _container = container;
            Files = new List<FileItem>();
        }

        public List<FileItem> Files { get; set; }

        public override Task ExecutePostProcessingAsync()
        {
            // Upload the files to azure blob storage and remove them from local disk
            foreach (var fileData in this.FileData)
            {
                var sp = new Stopwatch();
                sp.Start();
                string fileName = Path.GetFileName(fileData.Headers.ContentDisposition.FileName.Trim('"'));
                CloudBlockBlob blob = _container.GetBlockBlobReference(fileName);
                blob.Properties.ContentType = fileData.Headers.ContentType.MediaType;

                //set the number of blocks that may be simultaneously uploaded
                var requestOption = new BlobRequestOptions()
                {
                    ParallelOperationThreadCount = 5,
                    SingleBlobUploadThresholdInBytes = 10 * 1024 * 1024 ////maximum for 64MB,32MB by default
                };

                //upload a file to blob
                blob.UploadFromFile(fileData.LocalFileName, options: requestOption);
                blob.FetchAttributes();

                File.Delete(fileData.LocalFileName);
                sp.Stop();
                Files.Add(new FileItem
                {
                    ContentType = blob.Properties.ContentType,
                    Name = blob.Name,
                    SizeInMB = string.Format("{0:f2}MB", blob.Properties.Length / (1024.0 * 1024.0)),
                    Path = blob.Uri.AbsoluteUri,
                    BlobUploadCostInSeconds = string.Format("{0:f2}s", sp.ElapsedMilliseconds / 1000.0)
                });
            }
            return base.ExecutePostProcessingAsync();
        }
    }

结果如下:

enter image description here

答案 2 :(得分:1)

在一次请求中从Web API获取所有文件的SAS令牌后,我会检查将媒体直接上传到blob存储。使用来自客户端的promise和http get上传文件,这将并行化上传。 这将是您正确的设计和方法。这也将提高您的上传速度并减少延迟。