使用bash搜索日志文件的值

时间:2013-03-28 13:08:21

标签: bash file-io scripting logparser

我正在尝试创建一个脚本,该脚本将查询日志文件(或多个文件)中的特定值,并使用给定的值打印出来的不仅是它所在的行,而且只要它们是围绕它的所有行不是空白。

基本上我想像这样运行它:

logSearch.sh -f FilenameToSearch.log -s SearchTerm -b“20130323 12:00:00”e-“20130323 14:21:00”

FilenameToSearch.log包含以下内容:

03-10-2013 12:11:30
JunkData
JunkData

03-23-2013 12:00:00
JunkStill
Since
ValueLooking
ForIs
NotHere

03-23-2013 12:10:00
NotJunk
SearchTerm
ValueHere
NeedTo
GetAll
Yay

03-23-2013 12:10:30
BackToJunk
Blah

03-23-2013 12:11:00
SearchTerm
MorePrint

03-23-2013 15:10:00
SearchTerm
ButAfterGiven
Time
So
Junk

它将搜索日志文件,并在给定的日期时间值之间找到与第一个搜索词匹配的任何内容(它们是可选的)。

因此它将返回以下内容并将其传递给新文件:

03-23-2013 12:10:00
NotJunk
SearchTerm
ValueHere
NeedTo
GetAll
Yay

03-23-2013 12:11:00
SearchTerm
MorePrint

我在这里有基本代码我还没有实际处理数据,所以你们都可以提供帮助的任何事情都会非常令人满意!我将在整个星期继续努力,随着时间的推移将其充实,但你有任何想法(因为yall可能更好地编写这些脚本)将是有用的

代码

#!/bin/bash -eu

sytax="Proper invocation is logSearch.sh -s search-term -f filename-directory [b - datetimevalue1] [e - datetimevalue2]";
necVal="Error. -s and -f need to be specified";

usage () { echo $sytax; exit 1;}
error () { echo $necVal; usage; exit 2;}

options='s:f:b:e:x'

while getopts $options option
do
    case $option in

        s)
            searchFor=$OPTARG
            ;;
        f)
            inFilesLike=$OPTARG
            ;;
        b)
            betweenThis=$OPTARG
            ;;
        e)
            andThis=$OPTARG
            ;;
        *)
            usage;
            ;;
    esac
done

shift $(($OPTIND - 1))

if [ -z ${searchFor:-} ] || [ -z ${inFilesLike:-} ]
then
    error;
else
    echo "Search for : " $searchFor;
    echo "in files like: " $inFilesLike;
fi

if [ ! -z ${betweenThis:-} ]
then
    echo "Starting search at first occurance of: " $betweenThis;
else
    echo "No value to start search from. Beginning from the start of file(s).";
fi

if [ ! -z ${andThis:-} ]
then
    echo "Ending search at first occurance of: " $andThis;
else
    echo "No value to end search at. Will search to the end of file(s).";
fi

#BEGIN CODE TO SEARCH THROUGH $INFILESLIKE FILES FOR PARTICULAR VALUES

1 个答案:

答案 0 :(得分:0)

这是我在您的示例输入上测试的Python 3程序:

#!/usr/bin/env python

import re

# TODO: use argparse to get these values from the command line
filename = "t.txt"
term = "SearchTerm"
start = "20130323 12:00:00"
end = "20130323 14:21:00"

lineCache = None # will be a list once we find the start time
lineMatch = False # will be True when the search term was found in a block

for line in open(filename):
    match = re.match('(\d{2})-(\d{2})-(\d{4})( \d{2}:\d{2}:\d{2})\n', line)
    if match: # got a timestamp
        time = ''.join(map(match.group, [3, 1, 2, 4])) # "YYYYMMDD HH:MM:SS"
        if time > end:
            break;
        if time >= start:
            lineCache = [line]
            lineMatch = False
    elif lineCache is not None: # got a regular text line
        if term in line:
            lineMatch = True
            print(''.join(lineCache), end='') # print earlier lines from block
            lineCache = []
        if lineMatch:
            print(line, sep='')
        else: # haven't found the search term yet; store line in case we do
            lineCache.append(line)