C#解码Base64中的大/大XML / CDATA上下文

时间:2012-10-13 09:26:37

标签: c# stream base64 out-of-memory buffer

我得到了一个非常大的XML (> 1GB),内容为CDATA,其中包含76个字符和CRLF的base64,最后一个内容填充了“=”且没有CRLF。

获得base64解码最自然的方法是使用XmlReader reader.ReadElementContentAsBase64(...)但是它停止在100Mb工作。 (= OOM)

我在搜索了一种流式传输到Base64转换的方法之后找到了这个。

string Original = "foo bar, this is an example";

byte[] ToBase64; string Decoded;

using ( MemoryStream ms = new MemoryStream() )  using ( CryptoStream cs = new CryptoStream( ms, new ToBase64Transform(),
                                                CryptoStreamMode.Write ) )  using ( StreamWriter st = new StreamWriter( cs ) )  {
    st.Write( Original );
    st.Flush();
    ToBase64 = ms.ToArray();  }

using ( MemoryStream ms = new MemoryStream( ToBase64 ) )  using ( CryptoStream cs = new CryptoStream( ms, new FromBase64Transform(), 
                                                    CryptoStreamMode.Read ) )  using ( StreamReader sr = new StreamReader( cs ) )      {
    Decoded = sr.ReadToEnd(); }

Console.WriteLine( Original );  Console.WriteLine( Encoding.Default.GetString( ToBase64 ) );  Console.WriteLine( Decoded ); 

此示例有一个字符串作为输入,但我需要转换此代码以处理文件并从文件中的位置开始/停止读取。

              CData Start                         CDataEnd
                  |                                 |
                  V                                 V
     

[xml .. [Envelope] ... [Body] base64 with 76 char + CRLF + padding“=”... [/ Body] ... [/ Envelope]

1 个答案:

答案 0 :(得分:0)

这是我能创造的最好的......     int readbytes = 0;

                long bytesToRead = (CDataEnd - CDataStart);
                using (CryptoStream cryptoStremFromBase64 = new CryptoStream(context.OutStream, new FromBase64Transform(FromBase64TransformMode.IgnoreWhiteSpaces), CryptoStreamMode.Write))
                {


                    byte[] bytebuffer = new byte[bytesToRead % 10485760];
                    readbytes = context.InStream.Read(bytebuffer, 0, bytebuffer.Length);
                    cryptoStremFromBase64.Write(bytebuffer, 0, bytebuffer.Length);

                    while (context.InStream.Position < CDataEnd)
                    {
                        WriteToLog("Context.Instream.Position = " + context.InStream.Position.ToString(), context, startingTime);
                        byte[] bytebuffer2 = new byte[10485760];
                        readbytes = context.InStream.Read(bytebuffer2, 0, bytebuffer2.Length);
                        cryptoStremFromBase64.Write(bytebuffer2, 0, bytebuffer2.Length);
                    }


                    cryptoStremFromBase64.Flush();

                }

Context.InStream是带有base64的SOAP context.Outstream是从base64转换后的正文输出

有什么方法可以改善这个吗?