jq - 将嵌套的JSON提取到多个数组中

时间:2016-04-14 03:35:23

标签: json jq

我在本文的底部有一些(编辑过的)JSON,这是在AWS中运行ec2-describe-instances的结果。示例JSON有2个数组,但这可能更大。我之前使用过jq,但是我无法抽出深深嵌套在JSON中的值。特别是,我感兴趣的是:

  • InstanceId [“Reservations”:[“Instances”:[“InstanceId”:
  • DeviceName [“Reservations”:[“Instances”:[“BlockDeviceMappings”:[“DeviceName”:
  • VolumeId [“Reservations”:[“Instances”:[“BlockDeviceMappings”:[“Ebs”:{“VolumeId”:
  • TAG1 Value / Key
  • TAG2 Value / Key

总结一下,我想要一个可以迭代的JSON数组,它会给我一个InstanceID,该实例上的标签和单个卷。您会注意到每个实例都附加了3个卷(可能更多或更少)。我希望能够单独迭代每一个。我希望最终结果如下:

[
{
"InstanceId": "i-11111111",
"DeviceName": "/dev/sda1",
"VolumeId": "vol-1111111a",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-11111111",
"DeviceName": "xvdf",
"VolumeId": "vol-1111111b",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-11111111",
"DeviceName": "xvdg",
"VolumeId": "vol-1111111c",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-22222222",
"DeviceName": "/dev/sda1",
"VolumeId": "vol-2222222a",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-22222222",
"DeviceName": "/dev/sdb",
"VolumeId": "vol-2222222b",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-22222222",
"DeviceName": "/dev/sdc",
"VolumeId": "vol-2222222c",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
}
]

这类似于我发布的另一个问题here on stackoverflow。我已经尝试了很多不同的组合,但无法让它发挥作用。任何帮助都非常感谢。

这是JSON:

{
  "Reservations": [
    {
      "OwnerId": "xx",
      "ReservationId": "xx",
      "Groups": [],
      "Instances": [
        {
          "Monitoring": {
            "State": "xx"
          },
          "PublicDnsName": "",
          "Platform": "xx",
          "State": {
            "Code": xx,
            "Name": "xx"
          },
          "EbsOptimized": xx,
          "LaunchTime": "xx",
          "PrivateIpAddress": "xx",
          "ProductCodes": [],
          "VpcId": "xx",
          "StateTransitionReason": "",
          "InstanceId": "i-11111111",
          "ImageId": "xx",
          "PrivateDnsName": "xx",
          "KeyName": "xx",
          "SecurityGroups": [
            {
              "GroupName": "xx",
              "GroupId": "xx"
            },
            {
              "GroupName": "xx",
              "GroupId": "xx"
            }
          ],
          "ClientToken": "xx",
          "SubnetId": "xx",
          "InstanceType": "xx",
          "NetworkInterfaces": [
            {
              "Status": "xx",
              "MacAddress": "xx",
              "SourceDestCheck": xx,
              "VpcId": "xx",
              "Description": "xx",
              "NetworkInterfaceId": "xx",
              "PrivateIpAddresses": [
                {
                  "PrivateDnsName": "xx",
                  "Primary": xx,
                  "PrivateIpAddress": "xx"
                }
              ],
              "PrivateDnsName": "xx",
              "Attachment": {
                "Status": "xx",
                "DeviceIndex": xx,
                "DeleteOnTermination": xx,
                "AttachmentId": "xx",
                "AttachTime": "xx"
              },
              "Groups": [
                {
                  "GroupName": "xx",
                  "GroupId": "xx"
                },
                {
                  "GroupName": "xx",
                  "GroupId": "xx"
                }
              ],
              "SubnetId": "xx",
              "OwnerId": "xx",
              "PrivateIpAddress": "xx"
            }
          ],
          "SourceDestCheck": xx,
          "Placement": {
            "Tenancy": "xx",
            "GroupName": "xx",
            "AvailabilityZone": "xx"
          },
          "Hypervisor": "xx",
          "BlockDeviceMappings": [
            {
              "DeviceName": "/dev/sda1",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-1111111a",
                "AttachTime": "xx"
              }
            },
            {
              "DeviceName": "xvdf",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-1111111b",
                "AttachTime": "xx"
              }
            },
            {
              "DeviceName": "xvdg",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-11111111c",
                "AttachTime": "xx"
              }
            }
          ],
          "Architecture": "xx",
          "RootDeviceType": "xx",
          "RootDeviceName": "xx",
          "VirtualizationType": "hvm",
          "Tags": [
            {
              "Value": "TAG1",
              "Key": "VALUE1"
            },
            {
              "Value": "TAG2",
              "Key": "VALUE2"
            },
            {
              "Value": "TAG3",
              "Key": "VALUE3"
            },
            {
              "Value": "TAG4",
              "Key": "VALUE4"
            },
            {
              "Value": "TAG5",
              "Key": "VALUE5"
            },
            {
              "Value": "TAG6",
              "Key": "Value6"
            }
          ],
          "AmiLaunchIndex": xx
        }
      ]
    },
    {
      "OwnerId": "xx",
      "ReservationId": "xx",
      "Groups": [],
      "Instances": [
        {
          "Monitoring": {
            "State": "xx"
          },
          "PublicDnsName": "",
          "Platform": "xx",
          "State": {
            "Code": xx,
            "Name": "xx"
          },
          "EbsOptimized": xx,
          "LaunchTime": "xx",
          "PrivateIpAddress": "xx",
          "ProductCodes": [],
          "VpcId": "xx",
          "StateTransitionReason": "",
          "InstanceId": "i-22222222",
          "ImageId": "xx",
          "PrivateDnsName": "xx",
          "KeyName": "xx",
          "SecurityGroups": [
            {
              "GroupName": "xx",
              "GroupId": "xx"
            },
            {
              "GroupName": "xx",
              "GroupId": "xx"
            }
          ],
          "ClientToken": "xx",
          "SubnetId": "xx",
          "InstanceType": "xx",
          "NetworkInterfaces": [
            {
              "Status": "xx",
              "MacAddress": "xx",
              "SourceDestCheck": xx,
              "VpcId": "xx",
              "Description": "xx",
              "NetworkInterfaceId": "xx",
              "PrivateIpAddresses": [
                {
                  "PrivateDnsName": "xx",
                  "Primary": xx,
                  "PrivateIpAddress": "xx"
                }
              ],
              "PrivateDnsName": "xx",
              "Attachment": {
                "Status": "xx",
                "DeviceIndex": xx,
                "DeleteOnTermination": xx,
                "AttachmentId": "xx",
                "AttachTime": "xx"
              },
              "Groups": [
                {
                  "GroupName": "xx",
                  "GroupId": "xx"
                },
                {
                  "GroupName": "xx",
                  "GroupId": "xx"
                }
              ],
              "SubnetId": "xx",
              "OwnerId": "xx",
              "PrivateIpAddress": "xx"
            }
          ],
          "SourceDestCheck": xx,
          "Placement": {
            "Tenancy": "xx",
            "GroupName": "xx",
            "AvailabilityZone": "xx"
          },
          "Hypervisor": "xx",
          "BlockDeviceMappings": [
            {
              "DeviceName": "/dev/sda1",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-2222222a",
                "AttachTime": "xx"
              }
            },
            {
              "DeviceName": "/dev/sdb",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-2222222b",
                "AttachTime": "xx"
              }
            },
            {
              "DeviceName": "/dev/sdc",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-2222222c",
                "AttachTime": "xx"
              }
            }
          ],
          "Architecture": "xx",
          "RootDeviceType": "xx",
          "RootDeviceName": "xx",
          "VirtualizationType": "hvm",
          "Tags": [
            {
              "Value": "TAG1",
              "Key": "VALUE1"
            },
            {
              "Value": "TAG2",
              "Key": "VALUE2"
            },
            {
              "Value": "TAG3",
              "Key": "VALUE3"
            },
            {
              "Value": "TAG4",
              "Key": "VALUE4"
            },
            {
              "Value": "TAG5",
              "Key": "VALUE5"
            },
            {
              "Value": "TAG6",
              "Key": "Value6"
            }
          ],
          "AmiLaunchIndex": xx
        }
      ]
    }
]

编辑:我尝试过的一个例子(这个不是为了简单地试图获得标签:

jq -r '.Reservations.Instances | map(({ InstanceId } + (.BlockDeviceMappings | add) + (.BlockDeviceMappings.Ebs | Add))| { InstanceId, DeviceName, VolumeId })' <(echo "$json_array_windows")

jq: error: Add/0 is not defined at <top-level>, line 1:
.Reservations.Instances | map(({ InstanceId } + (.BlockDeviceMappings | add) + (.BlockDeviceMappings.Ebs | Add))| { InstanceId, DeviceName, VolumeId })                                      
jq: 1 compile error

1 个答案:

答案 0 :(得分:4)

以下jq过滤器根据请求生成JSON对象流。 如果你真的需要一个JSON数组,那么只需将整个表达式包装在方括号中。

void printhex(int x, char y, char z) {
      printf("%x%x%x",x,y,z);
}

void printdecimal( int x, char y ,char z) {
      printf("%i%i%i",x,y,z);
}     

typedef void (*printer)(int x, char y ,char z);

printer hex = &printhex;
printer dec = &printdecimal;

这是输出(一旦输入JSON被更正):

.Reservations[] | .Instances[]
| { InstanceId }
  + (.BlockDeviceMappings[]
     | { DeviceName,
         "VolumeId": .Ebs.VolumeId } )
  + (.Tags
     | { "TAG1": ( map( select(.Value == "TAG1"))[] | .Key),
         "TAG2": ( map( select(.Value == "TAG2"))[] | .Key) } )