在.NET中序列化大型JSON并通过HTTP进行流式处理

时间:2017-05-29 13:11:35

标签: c# .net json serialization stream

我正在尝试将每个12列的900000条记录序列化为JSON对象,并在.NET中通过HTTP对其进行流式传输。我得到一个内存不足的异常,因为它似乎最初是创建对象然后流式传输(我通过调试应用程序实现了这一点)。到目前为止,这是我的代码:

public class JsonStreamingResult : ActionResult
{
    private IEnumerable itemsToSerialize;

    public JsonStreamingResult(IEnumerable itemsToSerialize)
    {
        this.itemsToSerialize = itemsToSerialize;
    }

    public override void ExecuteResult(ControllerContext context)
    {
        var response = context.HttpContext.Response;
        response.ContentType = "application/json";
        response.ContentEncoding = System.Text.Encoding.UTF8;
        JsonSerializer serializer = new JsonSerializer();

        object itm;

        using (StreamWriter sw = new StreamWriter(response.OutputStream))
            using (JsonTextWriter writer = new JsonTextWriter(sw))
            {
                writer.WriteStartArray();
                foreach (object item in itemsToSerialize)
                {
                    Newtonsoft.Json.Linq.JObject obj = Newtonsoft.Json.Linq.JObject.FromObject(item, serializer);
                    obj.WriteTo(writer);
                    writer.Flush();
                }

                writer.WriteEndArray();
            }
    }
}

public class HomeController : Controller
{

    public ActionResult Index()
    {
        return View();
    }

    public ActionResult streamMason(int id)
    {
        var masonItems = new List<object>();
        DataContext dc = new DataContext();
        var viewData = dc.Articles.GetPublishedArticles(null, id);

        int cnt = 0;

        for (int i = 0; i < 900000; i++)
        {
            masonItems.Add(new
                    {
                    Field1 = "test1",
                    Field2 = "test2",
                    Field3 = "test3",
                    Field4 = "test4",
                    Field5 = "test5",
                    Field6 = "test6",
                    Field7 = "test7",
                    Field8 = "test8",
                    Field9 = "test9",
                    Field10 = "test10",
                    Field11 = "test11",
                    Field12 = "test12"
                    });
        }

        return new JsonStreamingResult(masonItems);
    }
}

我的问题:是否有可能在生成对象时将其流式传输?

我的代码基于:Streaming large list of data as JSON format using Json.net

编辑:当我尝试使用较少的记录(最多约300000)进行序列化时,它会按预期工作。

1 个答案:

答案 0 :(得分:2)

为感兴趣的人找到解决方案并张贴:

只需在response.Flush()之后添加:writer.Flush()