如何解析多行记录(用awk?)

时间:2015-03-30 23:10:15

标签: regex linux bash awk sed

我试图弄清楚如何从\n\n分隔的多行记录中提取特定字段。

在这种情况下,它碰巧从apt-cache输出,类似于DEBIAN控制文件。请参阅apt-cache show "$package"

的输出
Package: caffeine
Priority: optional
Section: misc
Installed-Size: 641
Maintainer: Reuben Thomas <rrt@sc3d.org>
Architecture: all
Version: 2.8.3
Depends: python3:any (>= 3.3.2-2~), python3, gir1.2-gtk-3.0, gir1.2-appindicator3-0.1, python3-xlib, python3-pkg-resources, libnet-dbus-perl
Filename: pool/main/c/caffeine/caffeine_2.8.3_all.deb
Size: 58774
MD5sum: 4438db3f6d1cf43a4f4b49cc7f24cda0
SHA1: e748370ac5ccd7de6fc9466ce0451d2e90d179d4
SHA256: ae303b4e32949cc1e1af80df7217e3406291679e3f18fa8f78a5bbb97504c4f6
Description-en: Prevent the desktop becoming idle in full-screen mode
 Caffeine stops the desktop becoming idle when an application
 is running full-screen. A desktop indicator ‘caffeine-indicator’
 supplies a manual toggle, and the command ‘caffeinate’ can be used
 to prevent idleness for the duration of any command.
Description-md5: 7c14f8adc007b10f6ecafed36260bedb

Package: caffeine
Priority: optional
Section: misc
Installed-Size: 655
Maintainer: Reuben Thomas <rrt@sc3d.org>
Architecture: all
Version: 2.6+555~ubuntu14.04.1
Depends: python:any (<< 2.8), python:any (>= 2.7.5-5~), python, gir1.2-gtk-2.0, gir1.2-appindicator3-0.1, x11-utils, python-dbus
Filename: pool/main/c/caffeine/caffeine_2.6+555~ubuntu14.04.1_all.deb
Size: 58604
MD5sum: 1051c3f7d40d344f986bb632d7436849
SHA1: 5e5f622595e8cbba8fb7468b3cffe2914b0ba110
SHA256: 11c5bbf2d28dcda6a7b82872195f740f1f79521b60d3c9acea3037bf0ab3a60e
Description: Prevent the desktop becoming idle
 Caffeine allows the user to prevent the desktop becoming idle,
 either manually or when certain applications are run. This
 prevents screen-blanking, locking, suspending, and so on.
Description-md5: 738866350e5086e77408d7a9c7ffa59b

Package: caffeine
Status: install ok installed
Priority: optional
Section: misc
Installed-Size: 794
Maintainer: Isaiah Heyer <freshapplepy@gmail.com>
Architecture: all
Version: 2.4.1+478~raring1
Depends: dconf-gsettings-backend | gsettings-backend, python (>= 2.6), python-central (>= 0.6.11), python-xlib, python-appindicator, python-xdg, python-notify, python-kaa-metadata
Description: Caffeine
 A status bar application able to temporarily prevent the activation
 of both the screensaver and the "sleep" powersaving mode.
Description-md5: 1c29acf1ab0f2e6636db29fbde1d14a3
Homepage: https://launchpad.net/caffeine
Python-Version: >= 2.6

我想要的输出是每条记录一行,格式为apt-get download $pkg=$ver -a=$arch。基本上是可用包的安装命令列表......

到目前为止,我所获得的是apt-cache show "$package" | awk '/^Package: / { print $2 } /^Version: / { print $2 } /^Architecture: / { print $2 }' | xargs -n3 | awk '{printf "apt-get download %s=%s -a=%s\n", $1, $3, $2}'

这是实际输出:

apt-get download caffeine=2.8.3 -a=all
apt-get download caffeine=2.6+555~ubuntu14.04.1 -a=all
apt-get download caffeine=2.4.1+478~raring1 -a=all

这是所希望的,但它似乎只是因为在这个例子中字段的顺序是一致的。如果字段的顺序不同,它会破裂。

我可以在Python中使用面向对象进行解析,但是我很难在一个awk命令中完成此操作。我能看到正确执行此操作的唯一方法是将每个记录拆分为单独的tmp文件(使用拆分或沿着这些行的某些内容),然后单独解析每个文件(这很简单)。显然我真的想避免不必要的I / O,因为这似乎是awk装备精良的东西。任何awk专业人士都知道如何解决这个问题?我甚至可以使用Perl单行或使用bash,但我真的很想学习如何更好地利用awk。

2 个答案:

答案 0 :(得分:6)

$ package=sed
$ apt-cache show "$package" | awk '/^Package: /{p=$2} /^Version: /{v=$2} /^Architecture: /{a=$2} /^$/{print "apt-get download "p"="v" -a="a}' 
apt-get download sed=4.2.1-10 -a=amd64

如何运作

  • /^Package: /{p=$2}

    将包信息保存在变量p

  • /^Version: /{v=$2}

    将版本信息保存在变量v

  • /^Architecture: /{a=$2}

    将架构信息保存在变量a

  • /^$/{print "apt-get download "p"="v" -a="a}

    当我们到达一个空白行时,请以所需的格式打印出信息。

    我的apt-cache版本始终在每个包后输出一个空行。您的示例输出缺少最后一个空白行。如果您的apt-cache确实没有产生最后一个空白行,那么我们需要添加更多代码来补偿。

    作为一种风格问题,有些人可能更喜欢printfprint。在这种情况下,将以上内容替换为:

    /^$/{printf "apt-get download %s=%s -a=%s\n",v,p,a}' 
    

答案 1 :(得分:3)

我发现处理包含名称到值对的数据的最佳方法是创建这些对的数组,然后只按名称访问这些值:

$ cat tst.awk
BEGIN { RS=""; FS="\n" }
{
    delete n2v
    for (i=1;i<=NF;i++) {
        if ($i !~ /^ /) {
            name = gensub(/:.*/,"","",$i)
            value = gensub(/[^:]+:\s+/,"","",$i)
            n2v[name] = value
        }
    }
    printf "apt-get download %s=%s -a=%s\n",
        n2v["Package"], n2v["Version"], n2v["Architecture"]
}

$ awk -f tst.awk file
apt-get download caffeine=2.8.3 -a=all
apt-get download caffeine=2.6+555~ubuntu14.04.1 -a=all
apt-get download caffeine=2.4.1+478~raring1 -a=all

以上使用了几个gawk扩展,但如果需要,很容易适应任何awk。