X12阅读器(C#)性能

时间:2016-08-31 14:48:07

标签: c# performance

@Hans Passant - 这个问题详细介绍了我在上一篇文章中没有得到答案的问题。我想要的只是帮助解决这个问题,并且我不断对技术问题进行评论。这是荒唐的。我已经尽力在这个问题上非常详细和具体,这个问题是一个未解决的问题。如果需要采取措施,请删除我之前的问题。

@Hans Passant - 你已经有效地杀死了我所有关于我的问题的讨论。谢谢!主持人的工作应该是为了促进有用的讨论,而不是用技术性的方式来解决它。现在我的两个问题都已经死了,我没有答案,而且最重要的是!踢球者是如果我发布另一个问题,它将被标记为重复!

有什么办法可以改善x12_reader的性能吗?似乎是瓶颈的主要功能是read_line()和get_element(),它们都被调用了数百万次。

using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Windows.Forms;

namespace X12ReprocessApp {

public class segment {
    public String line_text = "";
    public String loop_id = "";
    public String id = null;

    ///<summary>
    ///Returns a string containing the element from the current segment.
    ///</summary>
    ///<param name="element_number">The element number that you want to return</param>
    public String get_element(int element_number) {
        int count = 0;
        int start_index = 0;
        int end_index = 0;
        int current_index = 0;

        while (count < element_number && current_index != -1) {
            current_index = line_text.IndexOf(x12_reader.element_delimiter, start_index);
            start_index = current_index + 1;
            count++;
        }

        if (current_index != -1) {
            end_index = line_text.IndexOf(x12_reader.element_delimiter, start_index);
            if (end_index == -1) end_index = line_text.Length;
            return line_text.Substring(start_index, end_index - start_index);
        } else {
            return "";
        }
    }

    ///<summary>
    ///Returns a decimal containing the element from the current segment.
    ///</summary>
    ///<param name="element_number">The element number that you want to return</param>
    public Decimal get_element_as_number(int element_number) {
        Decimal i = (Decimal.TryParse(get_element(element_number), out i)) ? i : 0;
        return i;
    }

    ///<summary>
    ///Return added total of the elements within the segment.
    ///</summary>
    ///<param name="start">Element number to start adding.</param>
    ///<param name="step">Increment of element position after each add.</param>
    public Decimal get_elements_total(int start, int step) {
        Decimal total = 0;
        for (int i = start; i <= get_length(); i += step) {
            total += get_element_as_number(i);
        }
        return total;
    }

    public String get_id() {
        if (id != null) return id;
        int index = line_text.IndexOf(x12_reader.element_delimiter);
        if (index >= 0) {
            id = line_text.Substring(0, index);
            return id;
        }
        return String.Empty;
    }

    public int get_length() {
        int count = 1;
        for (int i = 0; i < line_text.Length; i++) {
            if (line_text[i] == x12_reader.element_delimiter) count++;
        }
        return count;
    }

    ///<summary>
    ///Returns a segment with the selected element replaced with the string value passed.
    ///</summary>
    ///<param name="element_number">The element that you want to replace.</param>
    ///<param name="value">The string value with which you want to replace the element.</param>
    public segment replace_element(int element_number, String value) {
        String[] elements = line_text.Split(x12_reader.element_delimiter);
        if (element_number < elements.Length && element_number > 0) {
            elements[element_number] = value;
            segment return_segment = new segment();
            return_segment.line_text = String.Join(x12_reader.element_delimiter.ToString(), elements);
            return return_segment;
        }
        return this;
    }
}

public class transaction {
    public segment[] segments;

    ///<summary>
    //Returns true if the current transaction contains the id.
    ///</summary>
    ///<param name="id">The id to find.</param>
    ///<param name="in_loop">The loop to limit the search.</param>
    public bool contains_id(String id, String in_loop = null) {
        foreach (segment s in segments) {
            if (in_loop != null) {
                if (s.get_id() == id && s.loop_id == in_loop) return true;
            } else if (s.get_id() == id) return true;
        }
        return false;
    }

    ///<summary>
    ///Returns the full text of the transaction as a String.
    ///</summary>
    public String get_full_text() {
        return get_lines(0, segments.Length - 1);
    }

    ///<summary>
    ///Returns a String containing a section of a transaction.
    ///</summary>
    public String get_lines(int start, int end) {
        StringBuilder sb = new StringBuilder();
        if (end >= segments.Length) end = segments.Length - 1;
        if (start >= 0) {
            for (int i = start; i <= end; i++) {
                sb.Append(segments[i].line_text);
                sb.AppendLine(x12_reader.line_terminator.ToString());
            }
            return sb.ToString();
        }
        return "";
    }

    ///<summary>
    ///Returns the first segment of the id type requested.
    ///</summary>
    ///<param name="id">ID of the segment to return.</param>
    ///<param name="element_1">First element of the ID to return.</param>
    public segment get_segment_of_type(String id, String element_1 = null) {
        foreach (segment s in segments) {
            if (s.get_id() == id) {
                if (element_1 != null) {
                    if (element_1 == s.get_element(1)) {
                        return s;
                    }
                } else {
                    return s;
                }
            }
        }
        return null;
    }

    ///<summary>
    ///Returns a List containing only segments with the corresponding id.
    ///</summary>
    public List<segment> get_segments_of_type(String id, String element_1 = null) {
        List<segment> temp_segment_list = new List<segment>();
        foreach (segment s in segments) {
            if (s.get_id() == id) {
                if (element_1 != null) {
                    if (element_1 == s.get_element(1)) {
                        temp_segment_list.Add(s);
                    }
                } else {
                    temp_segment_list.Add(s);
                }
            }
        }
        return temp_segment_list;
    }
}

public class x12_reader {
    public static Char element_delimiter;
    public static Char line_terminator;
    public static Char sub_delimiter;
    public segment GE;
    public segment GS;
    public Dictionary<String, Int32> id_counts = new Dictionary<String, Int32>();
    public segment IEA;
    public segment ISA;
    private FileStream file_stream;
    private StreamReader stream_reader;
    private StringBuilder string_builder = new StringBuilder();

    ///<summary>
    ///Class created to read and manage X12/EDI files.
    ///<param name="input">Path to the x12 file to be processed.</param>
    public x12_reader(String input) {
        file_stream = File.Open(input, FileMode.Open, FileAccess.Read);
        stream_reader = new StreamReader(file_stream, System.Text.Encoding.UTF8, false, 4096);

        line_terminator = read_char_at_location(105 + byte_order_mark_offset());
        sub_delimiter = read_char_at_location(104 + byte_order_mark_offset());
        element_delimiter = read_char_at_location(103 + byte_order_mark_offset());
    }

    ///<summary>
    ///Returns the number for segments with the selected id.
    ///</summary>
    ///<param name="id">The segment id to count.</param>
    public int get_segment_count(String id) {
        if (id_counts.ContainsKey(id)) return id_counts[id];
        return 0;
    }

    ///<summary>
    ///Builds the transactions list within the x12_reader class.
    ///</summary>
    public IEnumerable<transaction> read_x12(ProgressBar pBar = null) {
        MethodInvoker m = new MethodInvoker(() => pBar.Maximum = (int)stream_reader.BaseStream.Length);
        MethodInvoker v = new MethodInvoker(() => pBar.Value = (int)stream_reader.BaseStream.Position);

        int update_count = 0;
        List<segment> segments = new List<segment>();
        segment s;
        String current_loop = "";
        transaction t = new transaction();

        pBar.Invoke(m);
        using (stream_reader) {
            while (!stream_reader.EndOfStream) {
                update_count++;
                if (update_count >= 150000) {
                    pBar.Invoke(v);
                    update_count = 0;
                }

                s = get_segment();
                switch (s.get_id()) {
                    case "ISA":
                        ISA = s;
                        ISA.line_text = ISA.line_text.Substring(byte_order_mark_offset(), ISA.line_text.Length - byte_order_mark_offset());
                        ISA.loop_id = current_loop;
                        break;

                    case "IEA":
                        IEA = s;
                        IEA.loop_id = current_loop;
                        break;

                    case "GS":
                        GS = s;
                        GS.loop_id = current_loop;
                        break;

                    case "GE":
                        GE = s;
                        GE.loop_id = current_loop;
                        break;

                    case "ST":
                        current_loop = "";
                        s.loop_id = current_loop;
                        segments.Clear();
                        t = new transaction();
                        segments.Add(s);
                        break;

                    case "SE":
                        current_loop = "";
                        s.loop_id = current_loop;
                        segments.Add(s);
                        t.segments = segments.ToArray();
                        yield return t;
                        break;

                    case "N1":
                        if (s.get_element(1) == "PR") current_loop = "1000A";
                        if (s.get_element(1) == "PE") current_loop = "1000B";
                        s.loop_id = current_loop;
                        segments.Add(s);
                        break;

                    case "LX":
                        current_loop = "2000";
                        s.loop_id = current_loop;
                        segments.Add(s);
                        break;

                    case "CLP":
                        current_loop = "2100";
                        s.loop_id = current_loop;
                        segments.Add(s);
                        break;

                    case "SVC":
                        current_loop = "2110";
                        s.loop_id = current_loop;
                        segments.Add(s);
                        break;

                    case "PLB":
                        current_loop = "";
                        s.loop_id = current_loop;
                        segments.Add(s);
                        break;

                    default:
                        s.loop_id = current_loop;
                        segments.Add(s);
                        break;
                }
            }
            pBar.Invoke(v);
        }
    }

    private int byte_order_mark_offset() {
        if (read_char_at_location(0) == 0xEF &&
            read_char_at_location(1) == 0xBB &&
            read_char_at_location(2) == 0xBF) {
            return 3;
        }
        return 0;
    }

    private segment get_segment() {
        segment segment = new segment();
        segment.line_text = read_line();
        increment_count_for_id(segment.get_id());
        return segment;
    }

    private void increment_count_for_id(String id) {
        int value = 0;
        if (!id_counts.TryGetValue(id, out value)) id_counts.Add(id, 0);
        id_counts[id]++;
    }

    private Char read_char_at_location(long location) {
        long old_location = file_stream.Position;
        file_stream.Position = location;
        if (file_stream.Position < file_stream.Length) {
            Char c = (char)file_stream.ReadByte();
            file_stream.Position = old_location;
            return c;
        }
        file_stream.Position = old_location;
        return '\0';
    }

    private String read_line() {
        string_builder.Clear();
        int n;
        while ((n = stream_reader.Read()) != -1) {
            if (n == line_terminator) return string_builder.ToString();
            string_builder.Append((char)n);
        }
        return string_builder.ToString();
    }
}
}
  1. 我不能让get_element()一次返回多个元素。该函数必须在一次返回一个元素。

  2. 我已经运行过探查器。我的阻塞点似乎是Substring和Stringbuilder。

  3. 我已经尝试减少在调用代码中调用get_element()和read_line()函数的次数。我不认为我可以进一步减少通话量。

  4. 使用示例:

    x12_reader xr = new x12_reader(path_to_x12);
    
    foreach (transaction t in xr.read_x12(_progressbar_all_processing)) {
            //Do something with the transaction
            foreach(segment s in t.segments){
                   if(s.get_id() == "ID"){
                       s.get_element(5);
                   }
            }
    }
    

    替代尝试get_element函数。这最终变慢了,但我可能以次优的方式做了一些事情。

    public string get_element_alt(int index) {
            var buffer = new StringBuilder();
            var counter = 0;
    
            using (var enumerator = line_text.GetEnumerator()) {
                while (enumerator.MoveNext()) {
                    if (enumerator.Current == x12_reader.element_delimiter) {
                        counter++;
                    } else if (counter == index) {
                        buffer.Append(enumerator.Current);
                    } else if (counter > index)
                        break;
                }
            }
            return buffer.ToString();
        }
    

    在任何人标记此问题并告诉我将其发布在Code Review中之前。我已经完成了这一切,我得到的只是对我编码风格的迂腐反应。

    在任何人标记这个问题之前并说有其他关于使代码运行更快的帖子我会暂停并注意到我非常具体我想要做什么以及我需要更快地执行什么样的任务。我不是在问一般问题&#34;我如何编写快速代码。&#34;

1 个答案:

答案 0 :(得分:0)

在上一个问题中尝试对我的提案进行以下修改。

我们的想法是缓存StringBuilder以避免创建数百万个这些对象。我发现很难相信这可能会比重复拨打IndexOfSubstring更糟糕。

StringBuiler buffer = new StringBuilder();

public string get_element(int index)
{
     var counter = -1;

     using (var enumerator = text_line.GetEnumerator())
     {
         buffer.Clear();

         while (enumerator.MoveNext())
         {
             if (enumerator.Current == x12_reader.element_delimiter)
             {
                 counter++;
             }
             else if (counter == index)
             {
                 buffer.Append(enumerator.Current);
             }
             else if (counter > index)
                 break;
        }
     }

     return buffer.ToString();
}