C#从服务器下载大文件,内存消耗较少

时间:2017-05-05 12:02:43

标签: c# asp.net-mvc stream filestream

我有一个内存大小为42 MB的大文件。我想以较少的内存消耗下载文件 控制器代码

public ActionResult Download()
{
    var filePath = "file path in server";
    FileInfo file = new FileInfo(filePath);
    Response.ContentType = "application/zip";                        
    Response.AppendHeader("Content-Disposition", "attachment; filename=folder.zip");                   
    Response.TransmitFile(file.FullName);
    Response.End(); 
}

使用 Stream

尝试使用alernative方法
public ActionResult Download()
{           
    string failure = string.Empty;
    Stream stream = null;
    int bytesToRead = 10000;


    long LengthToRead;
    try
    {
        var path = "file path from server";
        FileWebRequest fileRequest = (FileWebRequest)FileWebRequest.Create(path);
        FileWebResponse fileResponse = (FileWebResponse)fileRequest.GetResponse();

        if (fileRequest.ContentLength > 0)
            fileResponse.ContentLength = fileRequest.ContentLength;

        //Get the Stream returned from the response
        stream = fileResponse.GetResponseStream();

        LengthToRead = stream.Length;

        //Indicate the type of data being sent
        Response.ContentType = "application/octet-stream";

        //Name the file 
        Response.AddHeader("Content-Disposition", "attachment; filename=SolutionWizardDesktopClient.zip");
        Response.AddHeader("Content-Length", fileResponse.ContentLength.ToString());

        int length;
        do
        {
            // Verify that the client is connected.
            if (Response.IsClientConnected)
            {
                byte[] buffer = new Byte[bytesToRead];

                // Read data into the buffer.
                length = stream.Read(buffer, 0, bytesToRead);

                // and write it out to the response's output stream
                Response.OutputStream.Write(buffer, 0, length);

                // Flush the data
                Response.Flush();

                //Clear the buffer
                LengthToRead = LengthToRead - length;
            }
            else
            {
                // cancel the download if client has disconnected
                LengthToRead = -1;
            }
        } while (LengthToRead > 0); //Repeat until no data is read

    }
    finally
    {
        if (stream != null)
        {
            //Close the input stream                   
            stream.Close();
        }
        Response.End();
        Response.Close();
    }
    return View("Failed");
}

由于文件的大小,它会消耗更多内存,从而导致性能问题 在检查iis日志后,下载过程分别为42 mb和64 mb 提前致谢

5 个答案:

答案 0 :(得分:18)

更好的选择是使用FileResult而不是ActionResult:

使用此方法意味着您不必在服务之前将文件/字节加载到内存中。

public FileResult Download()
{
     var filePath = "file path in server";
     return new FilePathResult(Server.MapPath(filePath), "application/zip");
}

编辑:对于较大的文件,FilePathResult也会失败。

你最好的选择可能就是Response.TransmitFile()。我已经在较大的文件(GB)上使用它,并且在

之前没有问题
public ActionResult Download()
{

    var filePath = @"file path from server";

    Response.Clear();
    Response.ContentType = "application/octet-stream";
    Response.AppendHeader("Content-Disposition", "filename=" + filePath);

    Response.TransmitFile(filePath);

    Response.End();

    return Index();
}

来自MSDN:

  

将指定文件直接写入HTTP响应输出流,   没有在内存中缓冲它。

答案 1 :(得分:3)

尝试将Transfer-Encoding标头设置为chunked,并返回带有PushStreamContent的HttpResponseMessage。分块的传输编码意味着HTTP响应不具有Content-Length头,因此客户端必须将HTTP响应的块作为流进行解析。请注意,我从未遇到过没有处理传输编码分块的客户端(浏览器等)。您可以在以下链接中阅读更多内容。

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding

    [HttpGet]
    public async Task<HttpResponseMessage> Download(CancellationToken token)
    {
        var response = new HttpResponseMessage(System.Net.HttpStatusCode.OK)
        {
            Content = new PushStreamContent(async (stream, context, transportContext) =>
            {
                try
                {
                    using (var fileStream = System.IO.File.OpenRead("some path to MyBigDownload.zip"))
                    {
                        await fileStream.CopyToAsync(stream);
                    }
                }
                finally
                {
                    stream.Close();
                }
            }, "application/octet-stream"),
        };
        response.Headers.TransferEncodingChunked = true;
        response.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment")
        {
            FileName = "MyBigDownload.zip"
        };
        return response;
    }

答案 2 :(得分:3)

我有类似的问题,但我没有在本地磁盘上存档,我不得不从API下载它(我的MVC就像一个代理)。 关键是在MVC Action上设置Response.Buffer=false;。我认为@ JanusPienaar的第一个解决方案应该适用于此。 我的MVC行动是:

public class HomeController : Controller
{
    public async Task<FileStreamResult> Streaming(long RecordCount)
    {
        HttpClient Client;
        System.IO.Stream Stream;

        //This is the key thing
        Response.Buffer=false;

        Client = new HttpClient() { BaseAddress=new Uri("http://MyApi", };
        Stream = await Client.GetStreamAsync("api/Streaming?RecordCount="+RecordCount);
        return new FileStreamResult(Stream, "text/csv");
    }
}

我的测试WebApi(生成文件)是:

public class StreamingController : ApiController
{
    // GET: api/Streaming/5
    public HttpResponseMessage Get(long RecordCount)
    {
        var response = Request.CreateResponse();

        response.Content=new PushStreamContent((stream, http, transport) =>
        {
            RecordsGenerator Generator = new RecordsGenerator();
            long i;

            using(var writer = new System.IO.StreamWriter(stream, System.Text.Encoding.UTF8))
            {
                for(i=0; i<RecordCount; i++)
                {
                    writer.Write(Generator.GetRecordString(i));

                    if(0==(i&0xFFFFF))
                        System.Diagnostics.Debug.WriteLine($"Record no: {i:N0}");
                    }
                }
            });

            return response;
        }

        class RecordsGenerator
        {
            const string abc = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
            char[] Chars = new char[14];//Ceiling(log26(2^63))

            public string GetRecordString(long Record)
            {
                int iLength = 0;
                long Div = Record, Mod;

                do
                {
                    iLength++;
                    Div=Math.DivRem(Div, abc.Length, out Mod);
                    //Save from backwards
                    Chars[Chars.Length-iLength]=abc[(int)Mod];
                }
                while(Div!=0);

                return $"{Record} {new string(Chars, Chars.Length-iLength, iLength)}\r\n";
            }
        }
    }
}

如果RecordCount为100000000,则TestApi生成的文件为1.56 GB。 WebApi和MVC都没有消耗太多内存。

答案 3 :(得分:1)

您只需使用IIS启用HTTP下载即可查看此link

,您只需要返回文件的HTTP路径即可快速轻松地下载。

答案 4 :(得分:1)

https://campaigns.zoho.com/api/v2/createcampaign?campaignName=mynewone&from_email=sammoudi.maher@gmail.com&subject=testSubject&authtoken=6fd68951538f7489e2406ac6f5a59bbe&resfmt=json个帖子对我有用:

  

在某些情况下,您需要为服务器上某处或运行时生成的大文件提供下载选项。以下功能可用于下载任何大小的文件。有时下载大文件会引发OutOfMemoryException异常,显示“内存不足,无法继续执行程序”。因此,此功能还可以通过将文件拆分为1 MB的块(可以通过更改bufferSize变量进行自定义)来处理这种情况。

用法:

DownloadLargeFile("A big file.pdf", "D:\\Big Files\\Big File.pdf", "application/pdf", System.Web.HttpContext.Current.Response);

您可以通过右侧Rizwan Ansari更改“ application / pdf”

下载功能:

public static void DownloadLargeFile(string DownloadFileName, string FilePath, string ContentType, HttpResponse response)
    {
        Stream stream = null;

        // read buffer in 1 MB chunks
        // change this if you want a different buffer size
        int bufferSize = 1048576;

        byte[] buffer = new Byte[bufferSize];

        // buffer read length
        int length;
        // Total length of file
        long lengthToRead;

        try
        {
            // Open the file in read only mode 
            stream = new FileStream(FilePath, FileMode.Open, FileAccess.Read, FileShare.Read);

            // Total length of file
            lengthToRead = stream.Length;
            response.ContentType = ContentType;
            response.AddHeader("Content-Disposition", "attachment; filename=" + HttpUtility.UrlEncode(DownloadFileName, System.Text.Encoding.UTF8));

            while (lengthToRead > 0)
            {
                // Verify that the client is connected.
                if (response.IsClientConnected)
                {
                    // Read the data in buffer
                    length = stream.Read(buffer, 0, bufferSize);

                    // Write the data to output stream.
                    response.OutputStream.Write(buffer, 0, length);

                    // Flush the data 
                    response.Flush();

                    //buffer = new Byte[10000];
                    lengthToRead = lengthToRead - length;
                }
                else
                {
                    // if user disconnects stop the loop
                    lengthToRead = -1;
                }
            }
        }
        catch (Exception exp)
        {
            // handle exception
            response.ContentType = "text/html";
            response.Write("Error : " + exp.Message);
        }
        finally
        {
            if (stream != null)
            {
                stream.Close();
            }
            response.End();
            response.Close();
        }
    }