将文本文件行分为数据库的两个不同表

时间:2012-08-15 21:25:15

标签: c# arrays linq text-files

我很抱歉,如果我不清楚这一天,我想要做的是逐行采取文本文件,并根据行开始将确定将其发送到哪个表。我遇到的问题是找到最好(最有效)的方式来发送某些以单词或“[”开头的行。这是我正在加载的文本文件的示例,它是日志,并且Exception,Message和Source可以在文本文件的任何位置。以“[”开头的行转到(LogTable),以“Exception”开头的行将转到(ExceptionTable),但每个“Exception”都连接到前一个Log行,所以我也在寻找一种方法连接这两个,以便它们可以在数据库中链接。我试图遍历文件中的每一行,并确定使用linq查询需要使用哪个表并将更改保存到mySQL数据库。

[2012-07-05 00:01:07,008]  [INFO ] [MessageManager]  [3780] () [] [Starting     ProcessNewMessageEvent]
[2012-07-05 00:01:07,008]  [INFO ] [MessageManager]  [3780] () [] [Method: RegValue]
[2012-07-05 00:01:07,008]  [DEBUG] [MessageManager]  [3780] () [] [reg: InstallPath]

Exception: System.ServiceModel.EndpointNotFoundException
Message: There was no endpoint listening at that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details.
Source: mscorlib
[2012-07-04 23:55:59,598]  [INFO ] [MessageManager]  [6616] () [] [Method: RegValue]
[2012-07-04 23:55:59,598]  [DEBUG] [MessageManager]  [6616] () [] [reg: InstallPath]

这段代码没有循环它会执行第一个循环然后它会在最后一节中被“grougquery”查询捕获并且不会遍历整个“for”循环,所以当它到达“异常:”时它抛出一个例外。有没有比我在这里做的更有效的方法呢?我找不到一种不同的方式?再次感谢

                foreach (string s in filePaths)
                {
                  string[] lines = System.IO.File.ReadAllLines(s);

                  for (int i = 0; i < lines.Length; i++)
                  {
                      if (lines[i].Contains("Exception:"))
                      {

                          var exQuery = from exMessage in lines
                                        let logRecord = exMessage.Split(':')
                                        select new ExTable()

                                        {
                                            ExlogException = logRecord[i],
                                            ExlogMessage = logRecord[i + 1],
                                            ExlogSource = logRecord[i + 2],

                                        };
                          foreach (var item in exQuery)
                          {
                              exception ex = new exception();


                              ex.LogException = item.ExlogException;
                              ex.LogMessage = item.ExlogMessage;
                              ex.LogSource = item.ExlogSource;
                              ex.LogServerStackTrace = item.ExlogServerStackTrace;
                              ex.LogExRethrown = item.ExlogRethrown;

                          }

                      }
                      else if (lines[i].Contains("["))
                      {

                          var groupQuery = from date in lines
                                           let logRecord = date.Split('[', ']', '(', ')')
                                           select new OLog()

                                           {
                                               OlogDate = logRecord[1],
                                               OlogLevel = logRecord[3],
                                               OlogLogger = logRecord[5],
                                               OlogThread = logRecord[7],
                                               OlogProperty = logRecord[9],
                                               OlogMethod = logRecord[11],
                                               OlogException = logRecord[12],

                                           };


                          foreach (var item in groupQuery)
                          {
                              temp temp = new temp();

                              temp.logDate = item.OlogDate;
                              temp.logLevel = item.OlogLevel;
                              temp.logLogger = item.OlogLogger;
                              temp.logThread = item.OlogThread;
                              temp.logProperty = item.OlogProperty;
                              temp.logMethod = item.OlogMethod;
                              temp.logException = item.OlogException;

                              logEntity.temps.AddObject(temp);
                          }

                      }
                  }
                          logEntity.SaveChanges();
               }

3 个答案:

答案 0 :(得分:0)

使用SSIS进行集成,它将更容易,它可以帮助您拆分文本文件 http://social.technet.microsoft.com/wiki/contents/articles/3172.aspx

答案 1 :(得分:0)

你按行划分,然后按':',它看起来像你的异常跨越3行,因此你的行数组中有3个不同的值,所以你的第二个拆分不会按预期工作。

实现我认为你想要完成的工作的一种方法是迭代线数组,然后在遇到它们时处理这些情况,如下所示:

foreach (string s in filePaths) {
//goes through each line in each file
string[] lines = System.IO.File.ReadAllLines(s);

for (int i=0;i < lines.Length; i++)
{
    if (line[i].Contains("Exception:"))
    {
        // Handle line[i] as Exception, line[i+1] as Message, line[i+2] as Source
    }
    else if (line[i].Contains("["))
    {
        // Same as you had before
    }
}
// same as before

}

答案 2 :(得分:0)

以下是我认为你要做的事情...以一种比一种for循环更有条理的方式。

注意,这个例子是在LinqPad(linqpad.com)中运行的,我推荐使用linq代码。它给出了预期的结果。

void Main()
{
    string [] lines = {
"[2012-07-05 00:01:07,008]  [INFO ] [MessageManager]  [3780] () [] [Starting     ProcessNewMessageEvent]",
"[2012-07-05 00:01:07,008]  [INFO ] [MessageManager]  [3780] () [] [Method: RegValue]",
"[2012-07-05 00:01:07,008]  [DEBUG] [MessageManager]  [3780] () [] [reg: InstallPath]",
@"Exception: System.ServiceModel.EndpointNotFoundException
Message: There was no endpoint listening at that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details.
Source: mscorlib",
"[2012-07-04 23:55:59,598]  [INFO ] [MessageManager]  [6616] () [] [Method: RegValue]",
"[2012-07-04 23:55:59,598]  [DEBUG] [MessageManager]  [6616] () [] [reg: InstallPath]"
    };

       var exceptions = lines.Where(l => l.Contains("Exception:"))
                          .Select(line => 
                          { 
                             var items = line.Split(':');
                             var result = new  
                                          {
                                             OlogException = items.ElementAtOrDefault(1),
                                             OlogMessage =   items.ElementAtOrDefault(2),
                                             OlogSource = items.ElementAtOrDefault(3),
                                             OlogServerStackTrace = items.ElementAtOrDefault(4)
                                          };
                             return result;
                          });

    var groups = lines.Where(l => l.StartsWith("["))
                      .Select(line => 
                       { 
                          var items = line.Split('[', ']', '(', ')');
                          var result = new 
                                       {
                                          OlogDate = items.ElementAtOrDefault(1),
                                          OlogLevel = items.ElementAtOrDefault(3),
                                          OlogLogger = items.ElementAtOrDefault(5),
                                          OlogThread = items.ElementAtOrDefault(7),
                                          OlogProperty = items.ElementAtOrDefault(9),
                                          OlogMethod = items.ElementAtOrDefault(11),
                                          OlogException = items.ElementAtOrDefault(13)
                                        };
                          return result;
                       });

  exceptions.Dump();
  groups.Dump();


}