bash在巨大的日志文件中查找字符串

时间:2014-11-20 11:11:35

标签: bash logging awk

我有一个巨大的日志文件,其中包含超过100M的字符串。 它包含19个colulms:

time | date | host | user | domain | category   | source | port | URL | etc

示例:

time    date    host    user    domain  category    source  port    URL etc
2:10:21 18.11.2014  192.168.56.101  %username1% %domainname%    "many words"    stackoverflow.com   "80"    http://stackoverflow.com/   
2:10:22 18.11.2014  192.168.56.101  %username2% %domainname%    "done"  stackoverflow.com   "80"    http://stackoverflow.com/   
2:10:23 18.11.2014  192.168.56.101  %username3% %domainname%    "denied site"   stackoverflow.com   "80"    http://stackoverflow.com/   
2:10:24 18.11.2014  192.168.56.101  %username4% %domainname%    "suspicious"    stackoverflow.com   "80"    http://stackoverflow.com/   
2:10:25 18.11.2014  192.168.56.101  %username5% %domainname%    "uncategorized" stackoverflow.com   "80"    http://stackoverflow.com/   
2:10:26 18.11.2014  192.168.56.101  %username6% %domainname%    "denied site"   stackoverflow.com   "80"    http://stackoverflow.com/   
2:10:27 18.11.2014  192.168.56.101  %username7% %domainname%    "many words"    stackoverflow.com   "80"    http://stackoverflow.com/

当我尝试在列中查找字符串时,它看起来很糟糕:

user@stand-01:~/folder$cat file |awk '{FS=" ";print$6}'
category
"many
"done"
"denied
"suspicious"
"uncategorized"
"denied
"many

所以当我尝试第7次colunm时,它有来自另一个colunm的数据:

user@stand-01:~/folder$cat file |awk '{FS=" ";print$7}'
source
words"
stackoverflow.com
site"
stackoverflow.com
stackoverflow.com
site"
words"

我如何使用空间分隔符并且不要在引号中分隔文本?

谢谢,伊势

2 个答案:

答案 0 :(得分:1)

这是一个awk

awk -F\" 'NR>1{print $2}' file
many words
done
denied site
suspicious
uncategorized
denied site
many words

或者

awk -F\" 'NR>1{print FS$2FS}' file
"many words"
"done"
"denied site"
"suspicious"
"uncategorized"
"denied site"
"many words"

答案 1 :(得分:0)

这样的事可能有效

$ awk '$6 ~ /^"[^"]+"$/{print $6;next} $6 ~ /^"/{print $6, $7}' input
"many words"
"done"
"denied site"
"suspicious"
"uncategorized"
"denied site"
"many words"