从bash脚本中的大型JSON输出中提取字符串到数组中

时间：2019-06-28 19:47:56

标签： arrays json bash grep

该json文件位于同一目录中我脚本的旁边，并且在开头附近包含“带有值的标题字段”，请参阅示例JSON中的“ Item1，Item2和Item3”。每个项目部分的末尾都有一个“服务”部分，每个服务都有一个标题。这些服务可以是全部，也可以没有，这取决于原始商品的标题。

我要做的就是搜索一个服务标题...即“ STitle5” 如果存在的话...将项目的主标题（即Item3）弹出到数组中。根据下面的示例JSON和刚刚给出的示例，只有Item1和Item3会添加到数组中。

我曾尝试过使用正则表达式进行多种方式的尝试，但是似乎无法弄清楚如何返回并抓住特定的字符串（如果在紧随其后的地方找到了）。 JSON中可以有成千上万的条目。我真的不需要任何其他东西，所以我认为将JSON直接解析为文本是最简单的方法。

    [{"id_name": "Item1", "informational": {"values": ["werwe", "werwwe", "8", "ewrwrw", "werewrew", "64432.5390625", "64432.55859375", "64432.36328125", "werw werwerw", "2.6.32-754.6.3.el6.x86_64", "2.6.32-696.16.1.el6.x86_64", "2.6.32-696.3.1.el6.x86_64", "2.6.32-642.13.1.el6.x86_64"], "fields": ["werwerw", "erwrwr", "wewrewrer", "werrwrwer", "werwerwrw", "werwerewr", "werwrwr", "stuff", "vendor_product", "version"]}, "role": ["Application Server"], "cpu_cores": ["8"], "create_time": "2017-04-03 16:32:27.738432", "mod_timestamp": "2019-06-26T01:17:23.933103+00:00", "title": "Item1", "family": ["dfsfd"], "OS": ["dfdsfsf"], "sdfdsfdsds": "fdsfdsf", "dsfdsfsd": ["64432.5390625", "64432.55859375", "64432.36328125"], "host": ["dfdsfsdfsdfds"], "sdfdsfds": "sdfdsf", "vend": ["sdada"], "permissions": {"delete": true, "write": true, "user": "dsdsds", "group": {"delete": true, "write": true, "read": true}, "read": true}, "sdsdsdsdsds": ["dfsdfdsfdsfsd"], "_version": "3", "sgrp": "default", "object_type": "dfsfs", "mod_by": "user", "mod_time": "2019-06-25 13:09:47.543535", "_user": "user", "environment": ["dfsdfdfsd"], "description": "", "identifier": {"values": ["dfsdfdfdsffdfsdfs"], "fields": ["host"]}, "sdfdsfsfds": ["SMP"], "role": ["operating_system_host"], "mod_source": "REST", "_key": "afderea-be2d-47a6-9f0d-00857ereef6c", "version": ["2.6.32-754.6.3.el6.x86_64", "2.6.32-696.16.1.el6.x86_64", "2.6.32-696.3.1.el6.x86_64", "2.6.32-642.13.1.el6.x86_64"], "create_source": "unknown", "services": [{"title": "STitle", "_key": "865defee-d47f-4b8f-9435-bc4ere89e9b1f8d"}, {"title": "STitle2", "_key": "d9d5e231-3841-4376-a295-ea5fere95168482"}, {"title": "STitle3", "_key": "38165ff4-9da6-df-9a8b-a162aa7a68e8"}, {"title": "S

“，” _key“：” e2adb75e-9254-4774-b735-“}，{” title“：” STitle6“，” _key“：” 381f54d0-d759-43a3-94b3“}，{” title“： “ STitle7”，“ _key”：“ 8253-f2b6a1d6f836”}，{“ title”：“ STitle8”，“ _key”：“ bc69692b-48d8-4bd7-b62b”}]}}，

    {"id_name": "Item2", "informational": {"values": ["werwe", "werwwe", "8", "ewrwrw", "werewrew", "64432.5390625", "64432.55859375", "64432.36328125", "werw werwerw", "2.6.32-754.6.3.el6.x86_64", "2.6.32-696.16.1.el6.x86_64", "2.6.32-696.3.1.el6.x86_64", "2.6.32-642.13.1.el6.x86_64"], "fields": ["werwerw", "erwrwr", "wewrewrer", "werrwrwer", "werwerwrw", "werwerewr", "werwrwr", "stuff", "vendor_product", "version"]}, "role": ["Application Server"], "cpu_cores": ["8"], "create_time": "2017-04-03 16:32:27.738432", "mod_timestamp": "2019-06-26T01:17:23.933103+00:00", "title": "Item2", "family": ["dfsfd"], "OS": ["dfdsfsf"], "sdfdsfdsds": "fdsfdsf", "dsfdsfsd": ["64432.5390625", "64432.55859375", "64432.36328125"], "host": ["dfdsfsdfsdfds"], "sdfdsfds": "sdfdsf", "vend": ["sdada"], "permissions": {"delete": true, "write": true, "user": "dsdsds", "group": {"delete": true, "write": true, "read": true}, "read": true}, "sdsdsdsdsds": ["dfsdfdsfdsfsd"], "_version": "3", "sgrp": "default", "object_type": "dfsfs", "mod_by": "user", "mod_time": "2019-06-25 13:09:47.543535", "_user": "user", "environment": ["dfsdfdfsd"], "description": "", "identifier": {"values": ["dfsdfdfdsffdfsdfs"], "fields": ["host"]}, "sdfdsfsfds": ["SMP"], "role": ["operating_system_host"], "mod_source": "REST", "_key": "afderea-be2d-47a6-9f0d-00857ereef6c", "version": ["2.6.32-754.6.3.el6.x86_64", "2.6.32-696.16.1.el6.x86_64", "2.6.32-696.3.1.el6.x86_64", "2.6.32-642.13.1.el6.x86_64"], "create_source": "unknown", "services": [{"title": "STitle", "_key": "865defee-d47f-4b8f-9435-bc4ere89e9b1f8d"}, {"title": "STitle2", "_key": "d9d5e231-3841-4376-a295-ea5fere95168482"}]}, 

    {"id_name": "Item3", "informational": {"values": ["werwe", "werwwe", "8", "ewrwrw", "werewrew", "64432.5390625", "64432.55859375", "64432.36328125", "werw werwerw", "2.6.32-754.6.3.el6.x86_64", "2.6.32-696.16.1.el6.x86_64", "2.6.32-696.3.1.el6.x86_64", "2.6.32-642.13.1.el6.x86_64"], "fields": ["werwerw", "erwrwr", "wewrewrer", "werrwrwer", "werwerwrw", "werwerewr", "werwrwr", "stuff", "vendor_product", "version"]}, "role": ["Application Server"], "cpu_cores": ["8"], "create_time": "2017-04-03 16:32:27.738432", "mod_timestamp": "2019-06-26T01:17:23.933103+00:00", "title": "Item3", "family": ["dfsfd"], "OS": ["dfdsfsf"], "sdfdsfdsds": "fdsfdsf", "dsfdsfsd": ["64432.5390625", "64432.55859375", "64432.36328125"], "host": ["dfdsfsdfsdfds"], "sdfdsfds": "sdfdsf", "vend": ["sdada"], "permissions": {"delete": true, "write": true, "user": "dsdsds", "group": {"delete": true, "write": true, "read": true}, "read": true}, "sdsdsdsdsds": ["dfsdfdsfdsfsd"], "_version": "3", "sgrp": "default", "object_type": "dfsfs", "mod_by": "user", "mod_time": "2019-06-25 13:09:47.543535", "_user": "user", "environment": ["dfsdfdfsd"], "description": "", "identifier": {"values": ["dfsdfdfdsffdfsdfs"], "fields": ["host"]}, "sdfdsfsfds": ["SMP"], "role": ["operating_system_host"], "mod_source": "REST", "_key": "afderea-be2d-47a6-9f0d-00857ereef6c", "version": ["2.6.32-754.6.3.el6.x86_64", "2.6.32-696.16.1.el6.x86_64", "2.6.32-696.3.1.el6.x86_64", "2.6.32-642.13.1.el6.x86_64"], "create_source": "unknown", "services": [{"title": "STitle", "_key": "865defee-d47f-4b8f-9435-bc4ere89e9b1f8d"}, {"title": "STitle2", "_key": "d9d5e231-3841-4376-a295-ea5fere95168482"}, {"title": "STitle3", "_key": "38165ff4-9da6-df-9a8b-a162aa7a68e8"}, {"title": "SSTitle5", "_key": "e2adb75e-9254-4774-b735-"}, {"title": "STitle6", "_key": "381f54d0-d759-43a3-94b3"}, {"title": "STitle7", "_key": "8253-f2b6a1d6f836"}, {"title": "STitle8", "_key": "bc69692b-48d8-4bd7-b62b"}]}]

编辑：可以使用id_name代替标题

编辑：添加了更好的示例数据

            {"id_name": "Item3", "informational": {"values": ["werwe", "werwwe", "8", "ewrwrw", "werewrew", "64432.5390625", "64432.55859375", "64432.36328125", "werw werwerw", "2.6.32-754.6.3.el6.x86_64", "2.6.32-696.16.1.el6.x86_64", "2.6.32-696.3.1.el6.x86_64", "2.6.32-642.13.1.el6.x86_64"], "fields": ["werwerw", "erwrwr", "wewrewrer", "werrwrwer", "werwerwrw", "werwerewr", "werwrwr", "stuff", "vendor_product", "version"]}, "role": ["Application Server"], "cpu_cores": ["8"], "create_time": "2017-04-03 16:32:27.738432", "mod_timestamp": "2019-06-26T01:17:23.933103+00:00", "title": "Item3", "family": ["dfsfd"], "OS": ["dfdsfsf"], "sdfdsfdsds": "fdsfdsf", "dsfdsfsd": ["64432.5390625", "64432.55859375", "64432.36328125"], "host": ["dfdsfsdfsdfds"], "sdfdsfds": "sdfdsf", "vend": ["sdada"], "permissions": {"delete": true, "write": true, "user": "dsdsds", "group": {"delete": true, "write": true, "read": true}, "read": true}, "sdsdsdsdsds": ["enttitle"=Item1, 'hostname=myHostname"], "_version": "3", "sgrp": "default", "object_type": "dfsfs", "mod_by": "user", "mod_time": "2019-06-25 13:09:47.543535", "_user": "user", "environment": ["dfsdfdfsd"], "description": "", "identifier": {"values": ["dfsdfdfdsffdfsdfs"], "fields": ["host"]}, "sdfdsfsfds": ["SMP"], "role": ["operating_system_host"], "mod_source": "REST", "_key": "afderea-be2d-47a6-9f0d-00857ereef6c", "version": ["2.6.32-754.6.3.el6.x86_64", "2.6.32-696.16.1.el6.x86_64", "2.6.32-696.3.1.el6.x86_64", "2.6.32-642.13.1.el6.x86_64"], "create_source": "unknown", "services": [{"title": "STitle", "_key": "865defee-d47f-4b8f-9435-bc4ere89e9b1f8d"}, {"title": "STitle2", "_key": "d9d5e231-3841-4376-a295-ea5fere95168482"}, {"title": "STitle3", "_key": "38165ff4-9da6-df-9a8b-a162aa7a68e8"}, {"title": "SSTitle5", "_key": "e2adb75e-9254-4774-b735-"}, {"title": "STitle6", "_key": "381f54d0-d759-43a3-94b3"}, {"title": "STitle7", "_key": "8253-f2b6a1d6f836"}, {"title": "STitle8", "_key": "bc69692b-48d8-4bd7-b62b"},{"id_name": "Item3", "informational": {"values": ["werwe", "werwwe", "8", "ewrwrw", "werewrew", "64432.5390625", "64432.55859375", "64432.36328125", "werw werwerw", "2.6.32-754.6.3.el6.x86_64", "2.6.32-696.16.1.el6.x86_64", "2.6.32-696.3.1.el6.x86_64", "2.6.32-642.13.1.el6.x86_64"], "fields": ["werwerw", "erwrwr", "wewrewrer", "werrwrwer", "werwerwrw", "werwerewr", "werwrwr", "stuff", "vendor_product", "version"]}, "role": ["Application Server"], "cpu_cores": ["8"], "create_time": "2017-04-03 16:32:27.738432", "mod_timestamp": "2019-06-26T01:17:23.933103+00:00", "title": "Item3", "family": ["dfsfd"], "OS": ["dfdsfsf"], "sdfdsfdsds": "fdsfdsf", "dsfdsfsd": ["64432.5390625", "64432.55859375", "64432.36328125"], "host": ["dfdsfsdfsdfds"], "sdfdsfds": "sdfdsf", "vend": ["sdada"], "permissions": {"delete": true, "write": true, "user": "dsdsds", "group": {"delete": true, "write": true, "read": true}, "read": true}, "sdsdsdsdsds": ["enttitle"=Item2, 'hostname=myHostname"], "_version": "3", "sgrp": "default", "object_type": "dfsfs", "mod_by": "user", "mod_time": "2019-06-25 13:09:47.543535", "_user": "user", "environment": ["dfsdfdfsd"], "description": "", "identifier": {"values": ["dfsdfdfdsffdfsdfs"], "fields": ["host"]}, "sdfdsfsfds": ["SMP"], "role": ["operating_system_host"], "mod_source": "REST", "_key": "afderea-be2d-47a6-9f0d-00857ereef6c", "version": ["2.6.32-754.6.3.el6.x86_64", "2.6.32-696.16.1.el6.x86_64", "2.6.32-696.3.1.el6.x86_64", "2.6.32-642.13.1.el6.x86_64"], "create_source": "unknown", "services": [{"title": "STitle", "_key": "865defee-d47f-4b8f-9435-bc4ere89e9b1f8d"}, {"title": "STitle2", "_key": "d9d5e231-3841-4376-a295-ea5fere95168482"}, {"title": "STitle3", "_key": "38165ff4-9da6-df-9a8b-a162aa7a68e8"}, {"title": "SSTitle4", "_key": "e2adb75e-9254-4774-b735-"}, {"title": "STitle6", "_key": "381f54d0-d759-43a3-94b3"}, {"title": "STitle7", "_key": "8253-f2b6a1d6f836"}, {"title": "STitle8", "_key": "bc69692b-48d8-4bd7-b62b"},{"id_name": "Item3", "informational": {"values": ["werwe", "werwwe", "8", "ewrwrw", "werewrew", "64432.5390625", "64432.55859375", "64432.36328125", "werw werwerw", "2.6.32-754.6.3.el6.x86_64", "2.6.32-696.16.1.el6.x86_64", "2.6.32-696.3.1.el6.x86_64", "2.6.32-642.13.1.el6.x86_64"], "fields": ["werwerw", "erwrwr", "wewrewrer", "werrwrwer", "werwerwrw", "werwerewr", "werwrwr", "stuff", "vendor_product", "version"]}, "role": ["Application Server"], "cpu_cores": ["8"], "create_time": "2017-04-03 16:32:27.738432", "mod_timestamp": "2019-06-26T01:17:23.933103+00:00", "title": "Item3", "family": ["dfsfd"], "OS": ["dfdsfsf"], "sdfdsfdsds": "fdsfdsf", "dsfdsfsd": ["64432.5390625", "64432.55859375", "64432.36328125"], "host": ["dfdsfsdfsdfds"], "sdfdsfds": "sdfdsf", "vend": ["sdada"], "permissions": {"delete": true, "write": true, "user": "dsdsds", "group": {"delete": true, "write": true, "read": true}, "read": true}, "sdsdsdsdsds": ["enttitle"=Item3, 'hostname=myHostname"], "_version": "3", "sgrp": "default", "object_type": "dfsfs", "mod_by": "user", "mod_time": "2019-06-25 13:09:47.543535", "_user": "user", "environment": ["dfsdfdfsd"], "description": "", "identifier": {"values": ["dfsdfdfdsffdfsdfs"], "fields": ["host"]}, "sdfdsfsfds": ["SMP"], "role": ["operating_system_host"], "mod_source": "REST", "_key": "afderea-be2d-47a6-9f0d-00857ereef6c", "version": ["2.6.32-754.6.3.el6.x86_64", "2.6.32-696.16.1.el6.x86_64", "2.6.32-696.3.1.el6.x86_64", "2.6.32-642.13.1.el6.x86_64"], "create_source": "unknown", "services": [{"title": "STitle", "_key": "865defee-d47f-4b8f-9435-bc4ere89e9b1f8d"}, {"title": "STitle2", "_key": "d9d5e231-3841-4376-a295-ea5fere95168482"}, {"title": "STitle3", "_key": "38165ff4-9da6-df-9a8b-a162aa7a68e8"}, {"title": "SSTitle5", "_key": "e2adb75e-9254-4774-b735-"}, {"title": "STitle6", "_key": "381f54d0-d759-43a3-94b3"}, {"title": "STitle7", "_key": "8253-f2b6a1d6f836"}, {"title": "STitle8", "_key": "bc69692b-48d8-4bd7-b62b"}]}]

“ enttitle = ”字段实际上包含我需要的确切值。数据文件全是一行，没有换行符。

2 个答案:

答案 0 :(得分：0)

这似乎是一种回旋处，但我设法获得了想要的结果。我愿意接受更有效的建议。但是否则，这是我写的，以防万一有人需要做这样的事情：

简而言之：

  1. 从文件中获取所有JSON数据并删除特殊字符->存储在var中
  2. 使用“ id_name”作为分隔符将新的字符串数据拆分为数组
  3. 遍历数组并将每个元素添加到新数组中，仅在原始元素中存在STitle5的地方
  4. 浏览最新的数组并grep在“ id_name”和“ informational”（即商品名称）之间的模式，将其添加到新数组中

注意： ：我最终在开始时使用“ id_name：Item1”，而不是title：Item1...。我希望标题匹配，但标题会出现在一百万个地方现在，id_name至少等于title ...尽管不需要。我需要将来对此做更好的证明。

        #!/bin/bash
        G_MATCH=$(grep -oP '(?=id_name).+?(?=id_name|}(](?!,)))' file.json | tr -d '[",:{}[]].')

        # split result to array using first name key as delim
        foo=$G_MATCH
        tmp="$foo"
        while [[ -n $tmp ]]; do
            tail=${tmp%id_name*}
            AR_ALL=("${tmp#$tail}" "${AR_ALL[@]}")
            tmp="$tail"
        done

        SUBSTRING="STitle5"

        # create new array only with items containing the specified service name
        AR_MATCHES=()
        for i in "${AR_ALL[@]}"
        do
                TEMP_VAL=$(echo $i|grep -o "$SUBSTRING")
                if [ -z "$TEMP_VAL" ]
                 then
                  echo "not adding..."
                else
                 AR_MATCHES+=("$i")
                fi
        done

        # create new array only with title names from first "name" key per line
        AR_TITLES=()
        for i in "${AR_MATCHES[@]}"
        do
                TEMP_VAL=$(echo $i|grep -oP '(?<=id_name )(.*)(?= informational)')
                if [ -z "$TEMP_VAL" ]
                 then
                  echo "not adding..."
                else
                 AR_TITLES+=("$TEMP_VAL")
                fi
        done

        printf '%s\n' "${AR_TITLES[@]}"

输出：

        Item1
        Item3

答案 1 :(得分：0)

这就是您要尝试的全部吗？

$ awk '/"STitle5"/{gsub(/^"|",$/,"",$2); print $2}' file
Item1
Item3

$ arr=( $(awk '/"STitle5"/{gsub(/^"|",$/,"",$2); print $2}' file) )
$ echo "${arr[0]}"
Item1
$ echo "${arr[1]}"
Item3

更新：给定新的单行输入后，您可以使用GNU awk针对多字符RS执行此操作：

$ awk -v RS='{"id_name":' '/"SSTitle5"/{gsub(/^"|",$/,"",$1); print $1}' file
Item3
Item3

请注意，在新输入中，所有项目都命名为“ Item3”，“ STitle”已替换为“ SSTitle”。尽管有your comment，但示例数据中没有名为entitle的字段。

相关问题
最新问题