使用bash解析查找第一个唯一值的文件

时间:2011-04-21 18:47:30

标签: parsing bash file

我有一个csv文件,我试图在bash中解析。每行的第一个字段是格式为yyyy-mm-dd hh:mm:ss的时间戳。每10分钟产生六条线,我在下面添加了一个小样本。

我要做的是从每天获得前6行。每天的第一个条目可以在00:00:xx和00:10:xx之间的任何时间发生,因此“00:0”的grep不起作用。

2010-04-23 00:04:43,0.0,0,4666724,3217665,28866,28866,0.92,65,
2010-04-23 00:04:43,0.1,0,4666724,3217663,20832,20832,0.62,65,
2010-04-23 00:04:43,0.2,0,4666724,3217662,14702,14702,0.46,65,
2010-04-23 00:04:43,0.3,0,4666724,3217664,27739,27739,0.92,65,
2010-04-23 00:04:43,0.4,0,4666724,3217664,25105,25105,0.77,65,
2010-04-23 00:04:43,0.5,0,4666724,3217664,24546,24546,0.77,65,
2010-04-23 00:14:43,0.0,0,4666724,3217665,29226,29226,0.92,65,
2010-04-23 00:14:43,0.1,0,4666724,3217663,21552,21552,0.62,65,
2010-04-23 00:14:43,0.2,0,4666724,3217662,15422,15422,0.46,65,
2010-04-23 00:14:43,0.3,0,4666724,3217664,28459,28459,0.92,65,
2010-04-23 00:14:43,0.4,0,4666724,3217664,25825,25825,0.77,65,
2010-04-23 00:14:43,0.5,0,4666724,3217664,25266,25266,0.77,65,
2010-04-23 00:24:43,0.0,0,4666724,3217665,29586,29586,0.92,65,
2010-04-23 00:24:43,0.1,0,4666724,3217663,22272,22272,0.77,65,
等等 2010-04-24 00:05:02,0.0,0,4666724,3217701,71388,71388,2.31,65,
2010-04-24 00:05:02,0.1,0,4666724,3217701,70264,70264,2.31,65,
2010-04-24 00:05:02,0.2,0,4666724,3217700,61254,61254,2.00,65,
2010-04-24 00:05:02,0.3,0,4666724,3217701,71011,71011,2.31,65,
2010-04-24 00:05:02,0.4,0,4666724,3217701,68111,68111,2.15,65,
2010-04-24 00:05:02,0.5,0,4666724,3217702,69904,69904,2.31,65,

想法,评论? 鲍勃

4 个答案:

答案 0 :(得分:2)

eugene y的答案的awk版本

awk '
    $1 != date {count = 0; date = $1} 
    ++count <= 6 {print}
' filename

答案 1 :(得分:1)

它可以像使用2种模式的grep一样简单:

grep -e " 00:0" -e " 00:10" myFIle.csv

第一个模式将匹配00:0000:09,第二个模式会找到00:10

答案 2 :(得分:1)

使用Perl应该很容易:

perl -ane '$l = 0 if $F[0] ne $d; print if $l++ < 6; $d = $F[0]' file

答案 3 :(得分:1)

以下使用read和自定义IFS(=输入字段分隔符)设置将输入行拆分为日期时间字段,其余部分则从ISO日期中提取日期-time使用bash'子串运算符,然后基本上继续打印下N行。在echo的位置,您可能希望在结果上放置您正在执行的任何处理,因为read + echo不会完全保留输入。

function first_n_of_each_day() {
    local N="$1"
    local lastDateTime=""
    local I=0
    while IFS=',' read DATETIME OTHER ; do
        local DATE="${DATETIME:0:10}"
        if [ "$DATE" != "$lastDateTime" ] ; then
            I=0
            lastDateTime="$DATE"
        fi
        if [ $I -lt "$N" ] ; then
            let ++I
            # line matches:
            echo "$DATETIME,$OTHER"
        fi
    done
}
first_n_of_each_day 6 < file.csv