尽管在解析字节数据时出错,但如何继续循环

时间:2013-06-19 09:58:40

标签: c# .net parsing loops binaryreader

我的问题是这个问题的延续:(loop for reading different data types & sizes off very large byte array from file

我有一个存储在文件(rawbytes.txt或bytes.data)上的原始字节流,我需要解析并输出到CSV样式的文本文件。

原始字节的输入(当读取为字符/ long / int等时)看起来像这样:

A2401028475764B241102847576511001200C...

解析它应该看起来像:

OutputA.txt

(Field1,Field2,Field3) - heading

A,240,1028475764

OutputB.txt

(Field1,Field2,Field3,Field4,Field5) - heading

B,241,1028475765,1100,1200

OutputC.txt

C,...//and so on

本质上,它是一个十六进制转储式的字节输入,它是连续的,没有任何行终止符或需要解析的数据之间的间隙。如上所示,数据一个接一个地包含不同的数据类型。

这是我的代码片段 - 因为在任何字段中都没有逗号,并且不需要使用“”(即CSV包装器),我只是使用TextWriter来创建CSV样式的文本文件,如下所示:

if (File.Exists(fileName))
        {
        using (BinaryReader reader = new BinaryReader(File.Open(fileName, FileMode.Open)))
            {
        while (reader.BaseStream.Position != reader.BaseStream.Length)
            {
                inputCharIdentifier = reader.ReadChar();
                switch (inputCharIdentifier)
                     case 'A':

                        field1 = reader.ReadUInt64();
                        field2 = reader.ReadUInt64();
                        field3 = reader.ReadChars(10);
                        string strtmp = new string(field3);
                        //and so on
                        using (TextWriter writer = File.AppendText("outputA.txt"))
                        {
                            writer.WriteLine(field1 + "," + field2 + "," + strtmp); // +  
                        }
                        case 'B':
                        //code...

我的问题是基于以下事实:一些原始字节数据包含很难解析的空值 - 因为有一个未知个空字节数(或非空字节数,连续数据块之间的异常字节(如果数据块没有损坏,则每个字节以A,B或C开头)。

问题

那么,如果因为数据损坏或错误而可能出现错误,我如何添加默认情况或其他机制来继续循环?以下代码是否有效?

    inputCharIdentifier = reader.ReadChar();
    ...
    case default:
    //I need to know what to add here, instead of default 
    //(i.e. the case when the character could not be read)
    while (binReader.PeekChar() != -1)
    {
         filling = binReader.readByte();
         //filling is a single byte
         try {
             fillingChar = Convert.ToChar(filling);

             break;
         }
         catch (Exception ex) { break; }
         if (fillingChar == 'A' || fillingChar == 'B')
             break;

剩下的部分 - 在没有停止程序的情况下向每个开关案例(例如'A')添加代码以继续 - 是否有办法在没有多个try-catch块的情况下执行此操作? [即代码块字符标识符是A,但A之后的字节是损坏的 - 在这种情况下我需要退出循环或读取(即跳过)一定数量的字节 - 如果消息头正确识别,这里将是已知的剩余的字节。

[注意:情况A,B等具有不同大小的输入 - 换句话说,A可能总共40个字节,而B是50个字节。因此,使用固定大小的缓冲区,例如inputBuf [1000]或[50] - 如果它们的大小相同 - 也不会很好用,AFAIK。]

有什么建议吗?请帮忙!我对C#相对较新(2个月)...

更新:我的整个代码如下:

         class Program
{
    const string fileName = "rawbytes.txt";
    static void Main(string[] args)
    {
                    try
        {
            var program = new Program();
            program.Parser();
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
        }
        Console.ReadLine();
    }
    public void Parser()
    {
        char inputCharIdentifier = 'Z';
        //only because without initializing inputCharIdentifier I ended up with an error
        //note that in the real code, 'Z' is not a switch-case alphabet
        //it's an "inconsequential character", i.e. i have defined it to be 'Z'
        //just to avoid that error, and to avoid leaving it as a null value
        ulong field1common = 0;
        ulong field2common = 0;
        char[] charArray = new char[10];
        char char1;
        char char2;
        char char3;
        int valint1 = 0;
        int valint2 = 0;
        int valint3 = 0;
        int valint4 = 0;
        int valint5 = 0;
        int valint6 = 0;
        int valint7 = 0;
        double valdouble;
        /*
        char[] filler = new char[53];
        byte[] filling = new byte[4621];
        byte[] unifiller = new byte[8];
        //these values above were temporary measures to manually filter through
        //null bytes - unacceptable for the final program
        */
        if (File.Exists(fileName))
        {
            using (BinaryReader reader = new BinaryReader(File.Open(fileName, FileMode.Open)))
            {
                while (reader.BaseStream.Position != reader.BaseStream.Length)
                {
                    //inputCharIdentifier = reader.ReadChar();
                    //if (inputCharIdentifier != null)
                    //{
                        try
                        {
                            inputCharIdentifier = reader.ReadChar();
                            try
                            {
                                switch (inputCharIdentifier)
                                {
                                    case 'A':

                                        field1common = reader.ReadUInt64();
                                        field2common = reader.ReadUInt64();
                                        //unifiller = reader.ReadBytes(8);
                                        //charArray = reader.ReadString();
                                        //result.ToString("o");
                                        //Console.WriteLine(result.ToString());
                                        charArray = reader.ReadChars(10);
                                        string charArraystr = new string(charArray);
                                        char1 = reader.ReadChar();
                                        valint1 = reader.ReadInt32();
                                        valint2 = reader.ReadInt32();
                                        valint3 = reader.ReadInt32();
                                        valint4 = reader.ReadInt32();
                                        using (TextWriter writer = File.AppendText("A.txt"))
                                        {
                                            writer.WriteLine(field1common + "," + /*result.ToString("o")*/ field2common + "," + charArraystr + "," + char1 + "," + valint1 + "," + valint2 + "," + valint3 + "," + valint4);
                                            writer.Close();
                                        }
                                        break;


                                    case 'B':
                                    case 'C':

                                        field1common = reader.ReadUInt64();
                                        field2common = reader.ReadUInt64();
                                        //charArray = reader.ReadString();
                                        charArray = reader.ReadChars(10);
                                        string charArraystr2 = new string(charArray);
                                        char1 = reader.ReadChar();
                                        valint1 = reader.ReadInt32();
                                        valint2 = reader.ReadInt32();
                                        using (TextWriter writer = File.AppendText("C.txt"))
                                        {
                                            writer.WriteLine(field1common + "," + result2.ToString("o") + "," + charArraystr2 + "," + char1 + "," + valint1 + "," + valint2);
                                            writer.Close();
                                        }
                                        break;
                                    case 'S':
                                        //market status message
                                        field1common = reader.ReadUInt64();
                                        char2 = reader.ReadChar();
                                        char3 = reader.ReadChar();
                                        break;
                                    case 'L':
                                        filling = reader.ReadBytes(4);
                                        break;
                                    case 'D':
                                    case 'E':
                                        field1common = reader.ReadUInt64();
                                        field2common = reader.ReadUInt64();
                                        //charArray = reader.ReadString();
                                        charArray = reader.ReadChars(10);
                                        string charArraystr3 = new string(charArray);
                                        //char1 = reader.ReadChar();
                                        valint1 = reader.ReadInt32();
                                        valint2 = reader.ReadInt32();
                                        valint5 = reader.ReadInt32();
                                        valint7 = reader.ReadInt32();
                                        valint6 = reader.ReadInt32();
                                        valdouble = reader.ReadDouble();
                                        using (TextWriter writer = File.AppendText("D.txt"))
                                        {
                                            writer.WriteLine(field1common + "," + result3.ToString("o") + "," + charArraystr3 + "," + valint1 + "," + valint2 + "," + valint5 + "," + valint7 + "," + valint6 + "," + valdouble);
                                            writer.Close();
                                        }
                                        break;
                                    }
                            }
                            catch (Exception ex)
                            {
                                Console.WriteLine("Parsing didn't work");
                                Console.WriteLine(ex.ToString());
                                break;
                            }
                        }
                        catch (Exception ex)
                        {
                            Console.WriteLine("Here's why the character read attempt didn't work");
                            Console.WriteLine(ex.ToString());

                            continue;
                            //continue;
                        }
                    //}
                }
            }
            }
            }

我收到的错误如下:

    Here's why the character read attempt didn't work

    System.ArgumentException: The output char buffer is too small to contain the decoded characters, encoding 'Unicode (UTF-8)' fallback 'System.Text.DecoderReplacementFallback'.
    Parameter name: chars
    at System.Text.Encoding.ThrowCharsOverflow()
    at System.Text.Encoding.ThrowCharsOverflow(DecoderNLS decoder, Boolean nothingDecoded)
    at System.Text.UTF8Encoding.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, DecoderNLS baseDecoder)
    at System.Text.DecoderNLS.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, Boolean flush)
    at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex, Boolean flush)
    at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex)
    at System.IO.BinaryReader.InternalReadOneChar()
    at System.IO.BinaryReader.Read()
    at System.IO.BinaryReader.ReadChar()
    at line 69: i.e. inputCharIdentifier = reader.ReadChar();

更新:生成相同错误的示例文件位于以下链接: http://www.wikisend.com/download/106394/rawbytes.txt

特别注意连续数据块之间的8个意外空字节,即使数据块标题 - 即 inputCharIdentifier - 有效。此类标头后面的字节数始终不可预测并且通常会有所不同。我的问题是,当出现下一个可用的非损坏数据块时,我需要能够删除或跳过这种情况 - 在示例文件的情况下,最后一个(单个)数据块发生在8个不合适的空字节。

8个空字节可以在文件中找到,如下所示: 字节计数器:1056 第2行,第783栏(根据记事本++)

问题的关键在于8个空字节可以是任何大小--3,7,15,50等。它总是未知 - 作为数据损坏的直接结果。但与“传统”数据损坏不同,即在数据块内部固定数量的字节(例如50)可能不可读并因此可以跳过(通过确切的字节数) - 我面临的数据损坏包括:有效数据块之间的字节数未知。

1 个答案:

答案 0 :(得分:2)

您无法为这些情况分配案例,因为目标变量(inputCharIdentifier)为null;因此,在避免这些情况的条件下就足够了。我还包括一个try ... catch,只是为了完全确定(执行所有给定操作时的任何错误都会使代码自动跳转到下面的迭代)。

try
{
    using (BinaryReader reader = new BinaryReader(File.Open(fileName, FileMode.Open), Encoding.ASCII))
    {
        while (reader.BaseStream.Position != reader.BaseStream.Length)
        {
            inputCharIdentifier = reader.ReadChar();
            if(inputCharIdentifier != null)
            {
               switch (inputCharIdentifier)
                 case 'A':
                    field1 = reader.ReadUInt64();
                    field2 = reader.ReadUInt64();
                    field3 = reader.ReadChars(10);
                    string strtmp = new string(field3);
                    //and so on
                    using (TextWriter writer = File.AppendText("outputA.txt"))
                    {
                       writer.WriteLine(field1 + "," + field2 + "," + strtmp); 
                    }
                 case 'B':
                   //code...
            }
        }
    }
}
catch
{
}