从日志文件中汇总时间

时间：2016-02-11 12:19:36

标签： bash date awk sed

我需要一个脚本来处理文件并总结SLA探测器的中断。我的文件如下：

Jan 28 22:10:02 test-195869-st28 Jan 28 22:10:04 test-195869-st28 Jan 28 22:10:06 test-195869-st28 Jan 28 22:10:08 test-195869-st28 Jan 28 22:10:10 test-195869-st28 Jan 28 22:10:12 test-195869-st28 Jan 28 22:10:14 test-195869-st28 Jan 28 22:15:16 test-195869-st28 Jan 28 23:12:34 test-195869-st28 Jan 28 23:12:36 test-195869-st28 Jan 28 23:12:38 test-195869-st28 Jan 28 23:12:40 test-195869-st28 Jan 28 23:12:42 test-195869-st28 Jan 28 23:12:44 test-195869-st28 Jan 28 23:12:46 test-195869-st28 Jan 28 23:12:48 test-195869-st28 Jan 28 23:12:50 test-195869-st28 Jan 28 23:12:52 test-195869-st28 Jan 29 13:12:18 test-195869-st28 Jan 29 13:12:20 test-195869-st28 Jan 29 13:12:22 test-195869-st28 Jan 29 13:12:24 test-195869-st28 Jan 29 13:12:26 test-195869-st28 Jan 29 13:12:28 test-195869-st28 Jan 29 13:12:30 test-195869-st28 Jan 29 13:12:32 test-195869-st28 Jan 29 13:12:34 test-195869-st28 Jan 29 13:12:36 test-195869-st28

所需的输出如下

Jan 28 22:10:02 test-195869-st28  
...  
Jan 28 22:10:14 test-195869-st28

Jan 28 22:15:16 test-195869-st28

Jan 28 23:12:34 test-195869-st28  
...  
Jan 29 13:12:36 test-195869-st28

应该总结所有中断（单个30秒间隔内的所有中断都是单次中断）。但是，单个第二次中断应作为单个条目保留在日志中。

直到现在我正试图用awk从文件中读取：

 awk 'BEGIN{prevDt=0;}
 {
  getDate="date -d \""$1" "$2" "$3"\" \"+%s\""
  print "Get date:" (getDate);
      while ( ( getDate | getline date ) > 0 ) {
  Diff=date-prevDt
  prevDt=date;
  print "Difference: "Diff" "$1" "$2" "$3" "$12;
 }
      close(getDate);

  } END { print $date }'

我正在考虑将这种差异用于以后的比较。

谢谢

3 个答案:

答案 0 :(得分：2)

当时间戳之间有30秒的时间间隔重叠时，不清楚你想要采用什么逻辑，所以这里是如何用GNU awk创建一个时间戳数组并按你喜欢的方式使用它：

$ cat tst.awk
NR==FNR {
    year  = strftime("%Y")
    month = (match("JanFebMarAprMayJunJulAugSepOctNovDec",$1)+2)/3
    day   = $2
    time  = gensub(/:/," ","g",$3)
    secs  = mktime(year" "month" "day" "time)

    print year, month, day, time, "=>", secs
    outages[NR] = secs FS $0
    next
}
{
    # loop through "outages" starting at current
    # line number to find those with first value
    # (secs) within 30 of the current value.
}


$ awk -f tst.awk file
2016 1 28 22 10 02 => 1454040602
2016 1 28 22 10 04 => 1454040604
2016 1 28 22 10 06 => 1454040606
2016 1 28 22 10 08 => 1454040608
2016 1 28 22 10 10 => 1454040610
2016 1 28 22 10 12 => 1454040612
2016 1 28 22 10 14 => 1454040614
2016 1 28 22 15 16 => 1454040916
2016 1 28 23 12 34 => 1454044354
2016 1 28 23 12 36 => 1454044356
2016 1 28 23 12 38 => 1454044358
2016 1 28 23 12 40 => 1454044360
2016 1 28 23 12 42 => 1454044362
2016 1 28 23 12 44 => 1454044364
2016 1 28 23 12 46 => 1454044366
2016 1 28 23 12 48 => 1454044368
2016 1 28 23 12 50 => 1454044370
2016 1 28 23 12 52 => 1454044372
2016 1 29 13 12 18 => 1454094738
2016 1 29 13 12 20 => 1454094740
2016 1 29 13 12 22 => 1454094742
2016 1 29 13 12 24 => 1454094744
2016 1 29 13 12 26 => 1454094746
2016 1 29 13 12 28 => 1454094748
2016 1 29 13 12 30 => 1454094750
2016 1 29 13 12 32 => 1454094752
2016 1 29 13 12 34 => 1454094754
2016 1 29 13 12 36 => 1454094756

当您掌握了最终逻辑时，请注释当前的print行。

答案 1 :(得分：1)

这将做你想要的：

#!/bin/bash

LogFile="mylog.log"

function Get-LineDate(){
    local Line="$1"
    if [[ "$Line" != "" ]]; then
        local LineDate=$(grep -oP ".*(?= test)" <<< "$Line")
        local LineDate=$(date -d "$LineDate" +"%s")
        echo "$LineDate"
    fi
}

function Get-DateDiff(){
    local DateOne="$1"
    local DateTwo="$2"

    if [[ "$DateOne" != "" ]] && [[ "$DateTwo" != "" ]]; then
        local DateDiff=$(( $DateOne -$DateTwo))
        echo "$DateDiff"
    fi
}


count=0
while read -r line; do
    if [[ "$count" -eq 0 ]]; then
        prevline="$line"
        prevdate=$(Get-LineDate "$line")
        echo "$line"
        count="1"
    else 
        linedate=$(Get-LineDate "$line")
        datediff=$(Get-DateDiff "$linedate" "$prevdate")
        if [[ "$datediff" -ge 30 ]]; then
            echo "$prevline -> $line - Interval: $datediff (seconds)"
        fi
        prevline="$line"
        prevdate="$linedate"
    fi
done < $LogFile

完整摘要输出：

Jan 28 22:10:02 test-195869-st28
Jan 28 22:10:14 test-195869-st28 -> Jan 28 22:15:16 test-195869-st28 - Interval: 302 (seconds)
Jan 28 22:15:16 test-195869-st28 -> Jan 28 23:12:34 test-195869-st28 - Interval: 3438 (seconds)
Jan 28 23:12:52 test-195869-st28 -> Jan 29 13:12:18 test-195869-st28 - Interval: 50366 (seconds)

如果您希望确切输出，则需要将while循环/逻辑更改为：

   while read -r line; do
    if [[ "$count" -eq 0 ]]; then
        prevline="$line"
        prevdate=$(Get-LineDate "$line")
        LineArr+=("$line")
        echo "$line"
    else 
        linedate=$(Get-LineDate "$line")
        datediff=$(Get-DateDiff "$linedate" "$prevdate")
        if [[ "$datediff" -ge 30 ]]; then
            for item in "${LineArr[@]}"; do
                if  [[ "$item" == "$prevline" ]]; then
                    bit=1
                    break
                fi  
            done
            if [[ "$bit" != "1" ]]; then 
                echo "$prevline"
                LineArr+=("$prevline")
            else
                bit=""
            fi

            for item in "${LineArr[@]}"; do
                if  [[ "$item" == "$line" ]]; then
                    bit=1
                    break
                fi  
            done
            if [[ "$bit" != "1" ]]; then 
                echo "$line"
                LineArr+=("$line")
            else
                bit=""
            fi
        fi
        prevline="$line"
        prevdate="$linedate"
    fi
    count=$(( $count +1 ))
done < $LogFile

输出：

Jan 28 22:10:02 test-195869-st28
Jan 28 22:10:14 test-195869-st28
Jan 28 22:15:16 test-195869-st28
Jan 28 23:12:34 test-195869-st28
Jan 28 23:12:52 test-195869-st28
Jan 29 13:12:18 test-195869-st28

答案 2 :(得分：1)

以下是最终代码：

#!/bin/bash

LogFile="test"


function Get-LineDate(){
    local Line="$1"
    if [[ "$Line" != "" ]]; then
        local LineDate=$(grep -oP ".*(?= MGIM0-test)" <<< "$Line")
        local LineDate=$(date -d "$LineDate" +"%s")
        echo "$LineDate"
    fi
}

function Get-DateDiff(){
    local DateOne="$1"
    local DateTwo="$2"

    if [[ "$DateOne" != "" ]] && [[ "$DateTwo" != "" ]]; then
        local DateDiff=$(( $DateOne -$DateTwo))
        echo "$DateDiff"
    fi
}

count=0
interval=0
prev_int=0

while read -r line; do
    if [[ "$count" -eq 0 ]]; then
        prevline="$line"
        prevdate=$(Get-LineDate "$line")
       # echo "$line"
        count="1"
        start_int="$line"
    else 
        linedate=$(Get-LineDate "$line")
        datediff=$(Get-DateDiff "$linedate" "$prevdate")
        prev_int=$interval 
        interval=$(( $datediff + $interval ))
    #   echo "Interval: $interval Previnterval: $prev_int"
    if [[ "$interval" -gt "$(( $prev_int+30 ))" ]] && [[ "$prev_int" -eq "0" ]]; then
        echo "$start_int - Single seconds outage"
        start_int="$line"

    fi
    if [[ "$interval" -gt "$(( $prev_int+30 ))" ]]; then
        echo -e "$start_int \n ... \n$prevline"
#       echo "$start_int -> $prevline - Interval: $prev_int (seconds)
        start_int="$line"
        interval=0
        prev_int=0
        fi

 #       if [[ "$datediff" -ge 30 ]]; then
  #          echo "$prevline -> $line - Interval: $datediff (seconds)"
       # fi
        prevline="$line"
        prevdate="$linedate"
    fi

done < $LogFile

if [[ "$start_int" ==  "$prevline" ]]; then
 echo "$start_int - Single seconds outage"
else
 echo "$start_int -> $prevline"
fi

全部谢谢！

相关问题
最新问题