DeflateStream.ReadAsync(.NET 4.5 System.IO.Compression)具有不同的字节读取返回值,而不是等效的Read方法?

时间:2015-01-07 14:41:42

标签: c# .net async-await deflatestream

在将一些旧代码转换为在c#中使用async时,我开始发现DeflateStream的Read()和ReadAsync()方法的返回值变化存在问题。

我认为从同步代码转换为

bytesRead = deflateStream.Read(buffer, 0, uncompressedSize);

它的等效异步版

bytesRead = await deflateStream.ReadAsync(buffer, 0, uncompressedSize);

应始终返回相同的值。


查看添加到问题底部的更新代码 - 以正确的方式使用流 - 从而使初始问题无关紧要


我发现经过多次迭代后,这并不成立,在我的具体情况下,在转换的应用程序中导致了随机错误。

我在这里错过了什么吗?

下面是一个简单的repro案例(在控制台应用程序中),Assert将在迭代#412的ReadAsync方法中为我打破,给出如下所示的输出:

....
ReadAsync #410 - 2055 bytes read
ReadAsync #411 - 2055 bytes read
ReadAsync #412 - 453 bytes read
---- DEBUG ASSERTION FAILED ----

我的问题是,为什么DeflateStream.ReadAsync方法此时返回453个字节?

注意:这只发生在某些输入字符串中 - StringBuilder中的大量CreateProblemDataString内容是我能想到为此帖子构建字符串的最佳方式。

class Program
{
    static byte[] DataAsByteArray;
    static int uncompressedSize;

    static void Main(string[] args)
    {
        string problemDataString = CreateProblemDataString();
        DataAsByteArray = Encoding.ASCII.GetBytes(problemDataString);
        uncompressedSize = DataAsByteArray.Length;
        MemoryStream memoryStream = new MemoryStream();
        using (DeflateStream deflateStream = new DeflateStream(memoryStream, CompressionMode.Compress, true))
        {
            for (int i = 0; i < 1000; i++)
            {
                deflateStream.Write(DataAsByteArray, 0, uncompressedSize);
            }
        }

        // now read it back synchronously
        Read(memoryStream);

        // now read it back asynchronously
        Task retval = ReadAsync(memoryStream);
        retval.Wait();
    }

    static void Read(MemoryStream memoryStream)
    {
        memoryStream.Position = 0;
        using (DeflateStream deflateStream = new DeflateStream(memoryStream, CompressionMode.Decompress, true))
        {
            byte[] buffer = new byte[uncompressedSize];
            int bytesRead = -1;
            int i = 0;
            while (bytesRead > 0 || bytesRead == -1)
            {
                bytesRead = deflateStream.Read(buffer, 0, uncompressedSize);
                System.Diagnostics.Debug.WriteLine("Read #{0} - {1} bytes read", i, bytesRead);
                System.Diagnostics.Debug.Assert(bytesRead == 0 || bytesRead == uncompressedSize);
                i++;
            }
        }
    }

    static async Task ReadAsync(MemoryStream memoryStream)
    {
        memoryStream.Position = 0;
        using (DeflateStream deflateStream = new DeflateStream(memoryStream, CompressionMode.Decompress, true))
        {
            byte[] buffer = new byte[uncompressedSize];
            int bytesRead = -1;
            int i = 0;
            while (bytesRead > 0 || bytesRead == -1)
            {
                bytesRead = await deflateStream.ReadAsync(buffer, 0, uncompressedSize);
                System.Diagnostics.Debug.WriteLine("ReadAsync #{0} - {1} bytes read", i, bytesRead);
                System.Diagnostics.Debug.Assert(bytesRead == 0 || bytesRead == uncompressedSize);
                i++;
            }
        }
    }

    /// <summary>
    /// This is one of the strings of data that was causing issues. 
    /// </summary>
    /// <returns></returns>
    static string CreateProblemDataString()
    {
        StringBuilder sb = new StringBuilder();
        sb.Append("0601051081                                      ");
        sb.Append("                                                                       ");
        sb.Append("                         225021         0300420");
        sb.Append("34056064070072076361102   13115016017");
        sb.Append("5      192         230237260250   2722");
        sb.Append("73280296      326329332   34535535");
        sb.Append("7   3                                                                  ");
        sb.Append("                                                                    4");
        sb.Append("                                                                             ");
        sb.Append("                                                         50");
        sb.Append("6020009      030034045   063071076   360102   13");
        sb.Append("1152176160170   208206      23023726025825027227328");
        sb.Append("2283285   320321333335341355357   622005009      0");
        sb.Append("34053      060070      361096   130151176174178172208");
        sb.Append("210198   235237257258256275276280290293   3293");
        sb.Append("30334   344348350                                                     ");
        sb.Append("                                                         ");
        sb.Append("                                           ");
        sb.Append("                                                                                   ");
        sb.Append("                                     225020012014   046042044034061");
        sb.Append("075078   361098   131152176160170   208195210   230");
        sb.Append("231260257258271272283306      331332336   3443483");
        sb.Append("54    29                                                           ");
        sb.Append("                                                                      ");
        sb.Append("                                                   2");
        sb.Append("5      29                                                06      0");
        sb.Append("1                                                            178      17");
        sb.Append("4                                                   205                     2");
        sb.Append("05      195                                                   2");
        sb.Append("31                     231      23");
        sb.Append("7                                       01              01    0");
        sb.Append("2                                              260                     26");
        sb.Append("2                                                            274                     2");
        sb.Append("72      274                                       01              01    0");
        sb.Append("3           1   5      3 6     43 52    ");
        return sb.ToString();
    }
}

更新的代码将流程正确地读入缓冲区

输出现在看起来像这样:

...
ReadAsync #410 - 2055 bytes read
ReadAsync #411 - 2055 bytes read
ReadAsync PARTIAL #412 - 453 bytes read, offset for next read = 453
ReadAsync #412 - 1602 bytes read
ReadAsync #413 - 2055 bytes read
...


static void Read(MemoryStream memoryStream)
    {
        memoryStream.Position = 0;
        using (DeflateStream deflateStream = new DeflateStream(memoryStream, CompressionMode.Decompress, true))
        {
            byte[] buffer = new byte[uncompressedSize]; // buffer to hold known fixed size record.
            int bytesRead; // number of bytes read from Read operation
            int offset = 0; // offset for writing into buffer
            int i = -1; // counter to track iteration #
            while ((bytesRead = deflateStream.Read(buffer, offset, uncompressedSize - offset)) > 0)
            {
                offset += bytesRead;  // offset in buffer for results of next reading
                System.Diagnostics.Debug.Assert(offset <= uncompressedSize, "should never happen - because would mean more bytes read than requested.");
                if (offset == uncompressedSize) // buffer full, complete fixed size record in buffer.
                {
                    offset = 0; // buffer is now filled, next read to start at beginning of buffer again.
                    i++; // increment counter that tracks iteration #
                    System.Diagnostics.Debug.WriteLine("Read #{0} - {1} bytes read", i, bytesRead);
                }
                else // buffer still not full
                {
                    System.Diagnostics.Debug.WriteLine("Read PARTIAL #{0} - {1} bytes read, offset for next read = {2}", i+1, bytesRead, offset);
                }
            }
        }
    }

    static async Task ReadAsync(MemoryStream memoryStream)
    {
        memoryStream.Position = 0;
        using (DeflateStream deflateStream = new DeflateStream(memoryStream, CompressionMode.Decompress, true))
        {
            byte[] buffer = new byte[uncompressedSize]; // buffer to hold known fixed size record.
            int bytesRead; // number of bytes read from Read operation
            int offset = 0; // offset for writing into buffer
            int i = -1; // counter to track iteration #
            while ((bytesRead = await deflateStream.ReadAsync(buffer, offset, uncompressedSize - offset)) > 0)
            {
                offset += bytesRead;  // offset in buffer for results of next reading
                System.Diagnostics.Debug.Assert(offset <= uncompressedSize, "should never happen - because would mean more bytes read than requested.");
                if (offset == uncompressedSize) // buffer full, complete fixed size record in buffer.
                {
                    offset = 0; // buffer is now filled, next read to start at beginning of buffer again.
                    i++; // increment counter that tracks iteration #
                    System.Diagnostics.Debug.WriteLine("ReadAsync #{0} - {1} bytes read", i, bytesRead);
                }
                else // buffer still not full
                {
                    System.Diagnostics.Debug.WriteLine("ReadAsync PARTIAL #{0} - {1} bytes read, offset for next read = {2}", i+1, bytesRead, offset);
                }
            }
        }
    }

1 个答案:

答案 0 :(得分:4)

达米恩的评论是完全正确的。但是,你的错误很常见,恕我直言这个问题值得一个实际的答案,如果没有其他原因,除了帮助那些犯同样错误的人更容易找到问题的答案。

所以,要明确:

对于.NET中所有面向流的I / O方法都是如此,其中一个提供byte[]缓冲区并且该方法返回读取的字节数,您可以对此进行唯一的假设字节数是:

  1. 该数字不会大于您要求读取的最大字节数(即作为要读取的字节数传递给该方法)
  2. 该数字将为非负数,只要实际上有剩余数据需要读取,数字将大于0(当您到达流的末尾时将返回0)。
  3. 当使用这些方法中的任何一种进行读取时,你甚至不能指望相同的方法总是返回相同的字节数(取决于上下文...显然在某些情况下,这实际上是确定性的,但你仍然不应该依赖于),并且不能保证不同的方法,即使是那些从同一来源读取的方法,总是会返回与其他方法相同的字节数。

    由调用者将字节作为流读取,考虑返回值,指定每次调用读取的字节数,并以适合该特定字节流的任何方式重新组合这些字节。 / p>

    请注意,在处理Stream个对象时,您可以使用Stream.CopyTo()方法。当然,它只复制到另一个Stream对象。但在许多情况下,可以使用目标对象而不将其视为Stream。例如。您只想将数据写为文件,或者您想将其复制到MemoryStream,然后使用MemoryStream.ToArray()方法将其转换为字节数组(然后您可以访问它)关注在给定的读操作中已经读取了多少字节...当你到达数组时,所有这些字节都被读取了:))。