Question

我的要求是将zip格式的文件上传到某个目的地。

该文件需要以zip文件的形式HTTP发布到目标
POST必须是包含其他数据的多部分消息（我认为这与我的问题无关）
解决方案位于.Net Core

我有一些工作代码;但是，这需要我将输入流复制到内存流中进行压缩，从而将整个文件加载到内存中。

当文件大于可用内存时，这会成为问题，导致我的进程因OOM而终止。

这是代码：

using System;
using System.IO;
using System.IO.Compression;
using System.Net.Http;
using System.Net.Http.Headers;

namespace HttpZipPipe
{
    public class Program
    {
        private const string sourceUrl = "https://aacapps.com/lamp/sound/amy.wav";
        private const string destUrl = "http://httpbin.org/post";    

        public static Stream CreateZipStream(Stream inputStream, string zipEntryName)
        {
            var zipStream = new MemoryStream();
            using (var zipArchive = new ZipArchive(zipStream, ZipArchiveMode.Create, true))
            {
                var zipEntry = zipArchive.CreateEntry(zipEntryName);
                using (var zipEntryStream = zipEntry.Open())
                    inputStream.CopyTo(zipEntryStream);
            }
            zipStream.Seek(0, SeekOrigin.Begin);
            return zipStream;
        }

        static void Main(string[] args)
        {
            String responseMessage;
            var Client = new HttpClient();

            using (var httpStream = Client.GetStreamAsync(sourceUrl).Result)
            using (var zipStream = CreateZipStream(httpStream, "audio.wav"))
            using (var fileContent = new StreamContent(zipStream))
            {  
                var content = new MultipartFormDataContent();
                content.Add(fileContent, "file", "file.zip");
                // ignored additional content and headers for clarity...
                responseMessage = Client.PostAsync(destUrl, content).Result.Content.ReadAsStringAsync().Result;
            }
            Console.WriteLine(responseMessage);
        }
    }
}

问题是＆＃34; CreateZipStream＆＃34;方法，尤其是：

inputStream.CopyTo(zipEntryStream);

我怀疑这导致我的进程进入OOM。

有没有办法通过zip流将输入流传输到内容而无需复制内存中的全部内容？

Answer 1

我不想使用您的服务，因此我使用常规网络应用程序来使用Response进行测试，但这样的事情应该可行。你必须同时阅读和写作。

我还使用SharpZipLib作为nuget包找到了。

对于文件，您必须为数据添加边界和文件名（在SO中添加多部分消息）。关于如何做到这一点有一些SO帖子。

// Create the request
var request = WebRequest.Create("https://aacapps.com/lamp/sound/amy.wav");
request.UseDefaultCredentials = true;

var size = 1024 * 8;
var buffer = new byte[size];

// Create a POST request
// The four next rows I haven't tested but it's a regular POST you'll need to do
var sq = WebRequest.Create("http://httpbin.org/post");
sq.Method = "POST";
sq.ContentType = "application/octet-stream";

// Get the stream to write to
using (var res = sq.GetRequestStream())
{
    // Create zip using the httpbin request stream as writer
    using (var zip = new ZipOutputStream(res))
    {
        zip.SetLevel(3);

        // Create file entry
        var entry = new ZipEntry(ZipEntry.CleanName("amy.wav"));
        zip.PutNextEntry(entry);

        // Get the data stream
        using (var stream = request.GetResponse().GetResponseStream())
        {
            int numBytes;
            // Read the data until no more
            while ((numBytes = stream.Read(buffer, 0, size)) > 0)
            {
                // Write to the zip file buffer
                zip.Write(buffer, 0, size);
            }
        }
        zip.Close();
    }
}

// Didn't try these either but it's just reading the result if any
var resp = sq.GetResponse();
new StreamReader(resp.GetResponseStream()).ReadToEnd();

如何压缩文件并通过HTTP发送而无需写入磁盘或将完整文件加载到内存中？

1 个答案: