Question

如果这太基础，我道歉。我真的不是在找人做这项工作，而是指出我正确的方向。我有一个日志文件可以追溯到几年，我想提取信息，以确定在性能缓慢时需要花多长时间查找模式。我能够读取每一行，但无法读取前一行以获得时间。

日志文件如下：

~
Other Stuff
~
12/21/11 18:58:15 Inserting data into ST_ITEMS
ST_ITEMS Row: 10000 inserted at 12/21/11 19:40:06
ST_ITEMS Row: 20000 inserted at 12/21/11 20:05:58
ST_ITEMS Row: 30000 inserted at 12/21/11 20:37:03
ST_ITEMS Row: 40000 inserted at 12/21/11 20:59:25
ST_ITEMS Row: 50000 inserted at 12/21/11 21:17:43
ST_ITEMS Row: 60000 inserted at 12/21/11 21:54:47
12/21/11 21:59:24 Finished inserting data into  Staging Tables

~
Other Stuff
~

12/21/11 22:04:43 Inserting data into ST_ITEMS
ST_ITEMS Row: 10000 inserted at 12/21/11 22:38:53
ST_ITEMS Row: 20000 inserted at 12/21/11 23:06:33
ST_ITEMS Row: 30000 inserted at 12/21/11 23:33:03
ST_ITEMS Row: 40000 inserted at 12/22/11 00:05:38
ST_ITEMS Row: 50000 inserted at 12/22/11 00:45:59
ST_ITEMS Row: 60000 inserted at 12/22/11 01:12:42
ST_ITEMS Row: 70000 inserted at 12/22/11 01:40:02
ST_ITEMS Row: 80000 inserted at 12/22/11 02:14:23
ST_ITEMS Row: 90000 inserted at 12/22/11 03:04:15
ST_ITEMS Row: 100000 inserted at 12/22/11 03:47:13
ST_ITEMS Row: 110000 inserted at 12/22/11 04:36:21
ST_ITEMS Row: 120000 inserted at 12/22/11 05:44:47
ST_ITEMS Row: 130000 inserted at 12/22/11 06:28:24
ST_ITEMS Row: 140000 inserted at 12/22/11 07:10:55
ST_ITEMS Row: 150000 inserted at 12/22/11 07:35:16
12/22/11 07:40:28 Finished inserting data into  Staging Tables

~
Other Stuff
~

基本上，我想通过从上面一行中减去一行的日期/时间来计算每10000行所需的时间。我正在考虑Perl和Bash作为选项，但似乎Perl提供了更多的可能性。

PERL ＃！的/ usr / bin中/ perl的

use strict;
use warnings;

use Date::Parse;
use Date::Format;

my $start = "2007-11-17 12:50:22";
my $stop  = "2007-11-17 12:53:22";
my $diff  = str2time($stop) - str2time($start);

#printf "diff between %s and %s is %d seconds\n", $start, $stop, $diff;

open(LOG,"info_refresh_tvl.log.122111_185800") or die "Unable to open logfile:$!\n";
while(my $line = <LOG>){


        if ($line=~/inserted\b/)

        {
        #Pseudocode  
            #Parse time from Pervious Line
            #Parse time from Current Line
            #Calculate Difference of Time
                    #my $diff  = str2time($stop) - str2time($start);
            #printf "diff between %s and %s is %d seconds\n", $start, $stop,     $diff; ')

            printf $line ;


        }

}
close(LOG);

BASH

grep 'ST_ITEMS Row:' logfile122111.log | while read line
   do
        event=$(echo "$line" | awk '{print $6 " " $7}')

       case $event in
          "10000")
                ;;
          *)
                past=$(echo "$line" | awk '{print $6 " " $7}')
                current=$(echo "$line" | awk '{print $6 " " $7}'
                echo $past
                echo $current)
                ;;
       esac



echo $event


   done

Answer 1

比较后继续保存每一行。完成后用当前行覆盖它。

在伪代码中：

$CurrentLine = $line;
#Parse time from $CurrentLine
#Parse time from $LastLine
#Calculate difference of time
$LastLine = $line;

Answer 2

正如其他人已经提到的那样，只需保留上一次参考时间。以下是使用Time::Piece的快速示例，这是自perl 5.10以来的核心模块：

use Time::Piece;

my $lasttime;
while(<DATA>) {
    chomp;

    my $diff;
    if(m{(\d+/\d+/\d+ \d+:\d+:\d+)}) {
        my $t = Time::Piece->strptime($1, "%D %H:%M:%S");
        if(defined $lasttime) {
            $diff = $t - $lasttime;
        }
        $lasttime = $t;
    }
    undef $lasttime if m{Finished inserting data};

    print "$_\t", ($diff && $diff->pretty) , "\n";
}

__DATA__
~
Other Stuff
~
12/21/11 18:58:15 Inserting data into ST_ITEMS
ST_ITEMS Row: 10000 inserted at 12/21/11 19:40:06
ST_ITEMS Row: 20000 inserted at 12/21/11 20:05:58
ST_ITEMS Row: 30000 inserted at 12/21/11 20:37:03
ST_ITEMS Row: 40000 inserted at 12/21/11 20:59:25
...

打印

~   
Other Stuff 
~   
12/21/11 18:58:15 Inserting data into ST_ITEMS  
ST_ITEMS Row: 10000 inserted at 12/21/11 19:40:06   41 minutes, 51 seconds
ST_ITEMS Row: 20000 inserted at 12/21/11 20:05:58   25 minutes, 52 seconds
ST_ITEMS Row: 30000 inserted at 12/21/11 20:37:03   31 minutes, 5 seconds
ST_ITEMS Row: 40000 inserted at 12/21/11 20:59:25   22 minutes, 22 seconds

Answer 3

grep -B1 gets the previous line before the line that is currently matched

Answer 4

您可以定义两个变量来保持时间。在伪代码中，将给出：

my $old = undef;
my $current;

while (my $line = <LOG>) {
    $line =~ /inserted at (.*)/ or next;
    $current = parse_time($1);
    if (defined $old) {
        printf("Time to insert 10k rows: %d\n", datediff($current, $old));
    }
    $old = $current;
}

（填补parse_time()和datediff()的空白，您应该设置）

日志文件的两行之间的时间差

4 个答案: