从FTP将文件下载到Azure存储的速度很慢C#

时间:2020-07-16 16:57:02

标签: c# azure

我当前正在从FTP下载文件并上传到Azure。我正在使用此link将大文件上传到Azure(使用BlockBlob)。

private async Task UploadFileAsBlockBlob(Stream stream, BlobStorageConfiguration blobStorageConfiguration, string fileName)
{
try
            {
                var storageAccount = CloudStorageAccount.Parse(blobStorageConfiguration.ConnectionString);
                var blobClient = storageAccount.CreateCloudBlobClient();
                var cloudContainer = blobClient.GetContainerReference(blobStorageConfiguration.Container);
                await cloudContainer.CreateIfNotExistsAsync().ConfigureAwait(false);

                var storageDirectory = cloudContainer.GetDirectoryReference(blobStorageConfiguration.Path);
                var blob = storageDirectory.GetBlockBlobReference(fileName);

                const long pageSizeInBytes = 1048576 * 10; // 10mb at a time

                var blocklist = new HashSet<string>();

                if (!stream.CanRead)
                {
                    var error = $"Error while reading file from ftp {fileName}";
                    return;
                }

                long prevLastByte = 0;
                long bytesRemain = stream.Length;

                do
                {
                    long bytesToCopy = Math.Min(bytesRemain, pageSizeInBytes);
                    byte[] bytesToSend = new byte[bytesToCopy];

                    // some issue might occur for very large files (above 2 gb)
                    // overflow exception will occur.
                    stream.Read(bytesToSend, 0, (int)bytesToCopy);

                    prevLastByte += bytesToCopy;
                    bytesRemain -= bytesToCopy;

                    // create blockId
                    string blockId = Guid.NewGuid().ToString();
                    string base64BlockId = Convert.ToBase64String(Encoding.UTF8.GetBytes(blockId));

                    await blob.PutBlockAsync(base64BlockId, new MemoryStream(bytesToSend, true), null).ConfigureAwait(false);

                    blocklist.Add(base64BlockId);
                }
                while (bytesRemain > 0);

                // post blocklist
                await blob.PutBlockListAsync(blocklist).ConfigureAwait(false);
            }
            catch (Exception ex)
            {
                Logger.LogCritical(ex, "Error while importing Files");
            }
}

我正在使用ftp上的Stream。 180Mb文件需要25分钟。建议使用更快的方法来加快速度吗?

0 个答案:

没有答案