读取日志文件并拉出包含特定文本的行

时间:2015-09-21 16:27:12

标签: c# regex wpf logfile-analysis

我有一个不断添加的日志(比如每秒1-5行),文件可能会变得相当大20MB +

示例日志文件:

[Wed Aug 26 00:01:48 2015] You try to kick a snake, but miss!
[Wed Aug 26 00:01:50 2015] A snake hits YOU for 3 points of damage.
[Wed Aug 26 00:01:51 2015] You punch a snake for 6 points of damage.
[Wed Aug 26 00:01:51 2015] A decaying skeleton tries to hit Clumsy, but misses!
[Wed Aug 26 00:01:53 2015] A snake hits YOU for 1 point of damage.
[Wed Aug 26 00:01:53 2015] A decaying skeleton tries to hit Clumsy, but misses!
[Wed Aug 26 00:01:54 2015] You gain experience!!
[Wed Aug 26 00:01:54 2015] You punch a snake for 6 points of damage.
[Wed Aug 26 00:01:54 2015] You have slain a snake!
[Wed Aug 26 00:01:56 2015] --You have looted a Snake Egg.--
[Wed Aug 26 00:01:56 2015] A decaying skeleton tries to hit Clumsy, but misses!
[Wed Aug 26 00:01:57 2015] --You have decided to not loot 2 item(s): Snake Fang. The item(s) will be available to anyone after the corpse(s) unlock.--
[Wed Aug 26 00:01:57 2015] --You have decided to not loot 2 item(s): Snake Scales. The item(s) will be available to anyone after the corpse(s) unlock.--
[Wed Aug 26 00:01:59 2015] a decaying skeleton's corpse splinters into hundreds of tiny bone fragments.
[Wed Aug 26 00:01:59 2015] Clumsy crushes a decaying skeleton for 7 points of damage.
[Wed Aug 26 00:01:59 2015] a decaying skeleton has been slain by Clumsy!
[Wed Aug 26 00:01:59 2015] Auto attack is on.
[Wed Aug 26 00:02:00 2015] Grug tells General:2, 'LFM AM/LORD'
[Wed Aug 26 00:02:00 2015] You punch a large rat for 5 points of damage.
[Wed Aug 26 00:02:00 2015] A large rat bites YOU for 1 point of damage.
[Wed Aug 26 00:02:00 2015] You kick a large rat for 1 point of damage.
[Wed Aug 26 00:02:02 2015] You punch a large rat for 2 points of damage.
[Wed Aug 26 00:02:03 2015] A large rat tries to bite YOU, but misses!
[Wed Aug 26 00:02:06 2015] You punch a large rat for 6 points of damage.
[Wed Aug 26 00:02:06 2015] A large rat bites YOU for 1 point of damage.
[Wed Aug 26 00:02:08 2015] a decaying skeleton clatters as it turns towards you.
[Wed Aug 26 00:02:08 2015] <SYSTEMWIDE_MESSAGE>: Gorenaire has been defeated by a group of hardy adventurers! Please join us in congratulating Kyrax along with everyone else who participated in this achievement!
[Wed Aug 26 00:02:08 2015] Clumsy tries to crush a decaying skeleton, but misses!
[Wed Aug 26 00:02:08 2015] A decaying skeleton tries to hit Clumsy, but misses!
[Wed Aug 26 00:02:09 2015] A large rat tries to bite YOU, but YOU dodge!
[Wed Aug 26 00:02:09 2015] You try to punch a large rat, but miss!
[Wed Aug 26 00:02:11 2015] Clumsy crushes a decaying skeleton for 8 points of damage.
[Wed Aug 26 00:02:11 2015] A decaying skeleton hits Clumsy for 2 points of damage.
[Wed Aug 26 00:02:11 2015] You kick a large rat for 1 point of damage.
[Wed Aug 26 00:02:12 2015] A large rat bites YOU for 1 point of damage.
[Wed Aug 26 00:02:13 2015] You gain experience!!
[Wed Aug 26 00:02:13 2015] You punch a large rat for 6 points of damage.
[Wed Aug 26 00:02:13 2015] You have slain a large rat!
[Wed Aug 26 00:02:13 2015] Clumsy crushes a decaying skeleton for 8 points of damage.
[Wed Aug 26 00:02:14 2015] A decaying skeleton hits Clumsy for 3 points of damage.
[Wed Aug 26 00:02:15 2015] --You have decided to not loot 1 item(s): Piece of Rat Fur. The item(s) will be available to anyone after the corpse(s) unlock.--
[Wed Aug 26 00:02:15 2015] --You have decided to not loot 2 item(s): Rat Whiskers. The item(s) will be available to anyone after the corpse(s) unlock.--
[Wed Aug 26 00:02:16 2015] a decaying skeleton's corpse splinters into hundreds of tiny bone fragments.
[Wed Aug 26 00:02:18 2015] Auto attack is on.
[Wed Aug 26 00:02:18 2015] a snake hisses and strikes!
[Wed Aug 26 00:02:18 2015] You punch a snake for 6 points of damage.
[Wed Aug 26 00:02:18 2015] A snake tries to hit YOU, but misses!
[Wed Aug 26 00:02:20 2015] You kick a snake for 1 point of damage.
[Wed Aug 26 00:02:21 2015] You try to punch a snake, but miss!
[Wed Aug 26 00:02:21 2015] A snake hits YOU for 4 points of damage.
[Wed Aug 26 00:02:24 2015] A snake tries to hit YOU, but misses!
[Wed Aug 26 00:02:25 2015] You try to punch a snake, but a snake parries!
[Wed Aug 26 00:02:27 2015] A snake tries to hit YOU, but misses!
[Wed Aug 26 00:02:28 2015] You kick a snake for 1 point of damage.
[Wed Aug 26 00:02:28 2015] You try to punch a snake, but miss!
[Wed Aug 26 00:02:30 2015] A snake hits YOU for 2 points of damage.
[Wed Aug 26 00:02:32 2015] You try to punch a snake, but miss!
[Wed Aug 26 00:02:33 2015] A snake tries to hit YOU, but misses!
[Wed Aug 26 00:02:36 2015] You punch a snake for 5 points of damage.
[Wed Aug 26 00:02:36 2015] A snake hits YOU for 1 point of damage.
[Wed Aug 26 00:02:37 2015] You kick a snake for 1 point of damage.
[Wed Aug 26 00:02:39 2015] A snake hits YOU for 1 point of damage.
[Wed Aug 26 00:02:40 2015] You gain experience!!
[Wed Aug 26 00:02:40 2015] You punch a snake for 6 points of damage.
[Wed Aug 26 00:02:40 2015] You have slain a snake!
[Wed Aug 26 00:02:42 2015] --You have looted a Snake Egg.--
[Wed Aug 26 00:02:42 2015] <SYSTEMWIDE_MESSAGE>: Innoruuk has been defeated by a group of hardy adventurers! Please join us in congratulating Harken along with everyone else who participated in this achievement!
[Wed Aug 26 00:02:43 2015] --You have decided to not loot 1 item(s): Snake Scales. The item(s) will be available to anyone after the corpse(s) unlock.--
[Wed Aug 26 00:02:46 2015] Auto attack is on.
[Wed Aug 26 00:02:46 2015] You punch a large rat for 6 points of damage.
[Wed Aug 26 00:02:46 2015] A large rat tries to bite YOU, but misses!
[Wed Aug 26 00:02:47 2015] You try to kick a large rat, but miss!
[Wed Aug 26 00:02:47 2015] You punch a large rat for 6 points of damage.
[Wed Aug 26 00:02:49 2015] A large rat bites YOU for 1 point of damage.
[Wed Aug 26 00:02:51 2015] You gain experience!!
[Wed Aug 26 00:02:51 2015] You punch a large rat for 5 points of damage.
[Wed Aug 26 00:02:51 2015] You have slain a large rat!
[Wed Aug 26 00:02:53 2015] --You have decided to not loot 1 item(s): Piece of Rat Fur. The item(s) will be available to anyone after the corpse(s) unlock.--
[Wed Aug 26 00:02:53 2015] --You have decided to not loot 2 item(s): Scalded Rat Skin. The item(s) will be available to anyone after the corpse(s) unlock.--
[Wed Aug 26 00:02:56 2015] Auto attack is on.
[Wed Aug 26 00:02:56 2015] You try to punch a large rat, but miss!
[Wed Aug 26 00:02:56 2015] A large rat bites YOU for 1 point of damage.
[Wed Aug 26 00:02:57 2015] You kick a large rat for 1 point of damage.

我将日志文件路径和名称存储在程序的设置中,如下所示:

Properties.Settings.Default.setting_logfolder // Folder Path
Properties.Settings.Default.setting_logfile // File Name

我需要打开文件,当我单击按钮并读取文件,直到再次单击按钮停止进程,同时正在读取文件我需要将它找到的行输出到文本框进行显示给用户。在发生这种情况时,父程序仍将写入日志文件。

这是我到目前为止按钮的代码。

    private void btnStart_Click(object sender, EventArgs e)
    {
        if (btnStart.Text == "Start Parsing")
        {
            // change text on button and switch status image
            btnStart.Text = "Stop Parsing";
            pbStatus.Image = Properties.Resources.online;

            string logfile = Properties.Settings.Default.setting_logfolder += Properties.Settings.Default.setting_logfile;

            if (File.Exists(logfile))
            {
                Stream stream = File.Open(logfile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
                StreamReader streamReader = new StreamReader(stream);
                string str = streamReader.ReadToEnd();
                tbOutput.AppendText(str);
                streamReader.Close();
                stream.Close();
            }
            else
            {
                tbOutput.AppendText(logfile);
            }
        }
        else
        {
            btnStart.Text = "Start Parsing";
            pbStatus.Image = Properties.Resources.offline;
        }
    }

我需要它继续读取文件,直到再次按下按钮,我只需要文件中的这样一行

[Wed Aug 26 00:02:08 2015] <SYSTEMWIDE_MESSAGE>: XXXXXXXXX has been defeated by a group of hardy adventurers! Please join us in congratulating XXXXXX along with everyone else who participated in this achievement!

4 个答案:

答案 0 :(得分:1)

我的答案是“for C#”

要查找包含系统范围消息的任何行,请使用:

\[[\w :]+\] <SYSTEMWIDE_MESSAGE>

对于该消息,但允许使用不同的名称,请使用:

\[[\w :]+\] <SYSTEMWIDE_MESSAGE>: \w+ has been defeated by a group of hardy adventurers! Please join us in congratulating \w+ along with everyone else who participated in this achievement!

这两者都假设用户/角色名称可以包含字母,数字或下划线

你当前的算法会执行一个ReadToEnd(),它不会读取单个行并且一次一个地咳嗽它们,这似乎是你想做的事情(?!也许!! ??!)。相反,它会读到流(文件)的末尾,然后咳嗽整个文件。

this answer to a related question中简要介绍了一个使用FileSystemWatcher的方法 - 对FileSystemWatcher有一些注意事项,但它应该是更“纯粹的.Net方法”

替代解决方案(Powershell)

如果您不需要GUI,并且您只想将这些信息放在屏幕上以便您可以对其进行一些操作,那么快速而肮脏的解决方案在PowerShell领域就是一行。打开PowerShell提示符(或创建脚本)并(假设我们使用的是一个幽默命名为chase.dat的日志文件和一个选定的正则表达式而不是<#regex#>,请使用以下代码:

cat chase.dat -tail 1 -wait | ?{$_ -match '<#regex#>'}

使用适当的正则表达式代替<#regex#>您的文件名代替chase.dat,它会吐出任何与您的正则表达式匹配的新行,因为它们是写入文件。如果你有兴趣,我可以解释一下......

<强> GUI!

如果你想组合GUI和PowerShell,上面的PS脚本行将传递给PipelineExecutor中概述的类中的“this article”构造函数,然后更新指定的属性(例如TextBox.Text)每当脚本返回更多(在本例中为已过滤)文本时。

刚刚做到这一点,它就像一个魅力!

答案 1 :(得分:1)

以下是使用FileWatcher Class

的简单方法
 public static Watchfile() 
 {
         FileSystemWatcher watch = new FileSystemWatcher();
         watch.Path = @"C:\";
         watch.Filter = "log.txt";
         watch.NotifyFilter = NotifyFilters.LastAccess |    
         NotifyFilters.LastWrite; 

        watch.Changed += new FileSystemEventHandler(OnChanged);
       watch.EnableRaisingEvents = true;
  }


 private static void OnChanged(object source, FileSystemEventArgs e)
 {


    string line;
         if (e.FullPath == @"C:\log.txt")
         {
             Regex _regex = new Regex(@"\[[\w :]+\] <SYSTEMWIDE_MESSAGE>: \w+ has been defeated by a group of hardy adventurers! Please join us in congratulating \w+ along with everyone else who participated in this achievement!");
             System.IO.StreamReader file =
                       new System.IO.StreamReader(@"C:\log.txt");

             while ((line = file.ReadLine()) != null)
             {
                 Match match = _regex.Match(line);
                 if (match.Success)
                 {

                     //Match Found

                 }


             }

             file.Close();
         }


 }

答案 2 :(得分:0)

好的,首先,如果您正在解析一个可能太大而无法保留在内存中的日志文件(大多数情况下不是20mb),那么请使用File.OpenText并读取它以块或线的形式出现。这意味着您不需要将整个文本文件加载到内存中。其次,我会将读取逻辑移除到它自己的类或函数中。如果您正在运行Windows窗体或WPF并希望在找到一行时更新UI,那么您需要在单独的线程/任务上执行此操作以使UI更新而不锁定它。使用Dispatcher类从后台任务更新UI。

然后,您的正则表达式需要基于您想要捕获的精确线条。在我看来,你可以在行的开头忽略方括号中的所有内容,然后断言:

\w+ has been defeated by a group of hardy adventurers! Please join us in congratulating \w+ along with everyone else who participated in this achievement!$

编辑:

从您的文件共享选项看,日志文件可能会增长吗?如果这是正确的,请避免逐行读取,因为您可以读取部分线。如果你在流中缓存你的位置,这可能意味着你失去了潜在的匹配。相反,读取一个块,检查Environment.NewLine,然后将其解析为正则表达式引擎中的一行。

答案 3 :(得分:-1)

试试这个:

-using (var fileStream = new FileStream("YOUR FILE", FileMode.Open, FileAccess.Read, FileShare.None))
{                  
    using (var streamReader = new StreamReader(fileStream))
    {
        string line;
        while((line = reader.ReadLine()) != null)
        {
          if(line == "YOUR TEXT")
          {
           // Do the rest
          }
        }
    }
}

您必须指定特定模式才能完成此示例。