Question

我正在与AWS EMR进行大量工作，当您通过AWS管理控制台构建EMR集群时，您可以单击一个按钮以导出创建EMR集群的AWS CLI命令。

然后，它会给您一个大型的CLI命令，该命令不会以任何方式格式化，即，如果您复制并粘贴该命令，则它们都在一行中。

我正在使用由其他人创建的这些EMR CLI命令，使用AWS开发工具包Boto3库在Python中创建EMR集群，即，我正在查看CLI命令以获取所有配置详细信息。一些配置详细信息显示在AWS管理控制台UI上，但不是全部，因此对于我来说，使用可以导出的CLI命令更容易。

但是，由于未格式化AWS CLI命令，因此很难阅读。网上是否有类似于JSON formatters的AWS CLI命令格式化程序？

我可以使用的另一种解决方案是克隆EMR集群，并通过AWS管理控制台上的EMR集群创建屏幕来获取所有配置详细信息，但是我仍然好奇是否可以格式化CLI命令并执行该操作办法。能够格式化导出的CLI命令的另一个附加好处是，我可以将其放在Confluence页面上以获取文档。

Answer 1

这里有一些快速的python代码可以做到这一点：

import shlex
import json
import re

def format_command(command):
    tokens = shlex.split(command)
    formatted = ''
    for token in tokens:
        # Flags get a new line
        if token.startswith("--"):
            formatted += '\\\n    '
        # JSON data
        if token[0] in ('[', '{'):
            json_data = json.loads(token)
            data = json.dumps(json_data, indent=4).replace('\n', '\n    ')
            formatted += "'{}' ".format(data)
        # Quote token when it contains whitespace
        elif re.match('\s', token):
            formatted += "'{}' ".format(token)
        # Simple print for remaining tokens
        else:
            formatted += token + ' '
    return formatted


example = """aws emr create-cluster --applications Name=spark Name=ganglia Name=hadoop --tags 'Project=MyProj' --ec2-attributes '{"KeyName":"emr-key","AdditionalSlaveSecurityGroups":["sg-3822994c","sg-ccc76987"],"InstanceProfile":"EMR_EC2_DefaultRole","ServiceAccessSecurityGroup":"sg-60832c2b","SubnetId":"subnet-3c76ee33","EmrManagedSlaveSecurityGroup":"sg-dd832c96","EmrManagedMasterSecurityGroup":"sg-b4923dff","AdditionalMasterSecurityGroups":["sg-3822994c","sg-ccc76987"]}' --service-role EMR_DefaultRole --release-label emr-5.14.0 --name 'Test Cluster' --instance-groups '[{"InstanceCount":1,"EbsConfiguration":{"EbsBlockDeviceConfigs":[{"VolumeSpecification":{"SizeInGB":32,"VolumeType":"gp2"},"VolumesPerInstance":1}]},"InstanceGroupType":"MASTER","InstanceType":"m4.xlarge","Name":"Master"},{"InstanceCount":1,"EbsConfiguration":{"EbsBlockDeviceConfigs":[{"VolumeSpecification":{"SizeInGB":32,"VolumeType":"gp2"},"VolumesPerInstance":1}]},"InstanceGroupType":"CORE","InstanceType":"m4.xlarge","Name":"CORE"}]' --configurations '[{"Classification":"spark-defaults","Properties":{"spark.sql.avro.compression.codec":"snappy","spark.eventLog.enabled":"true","spark.dynamicAllocation.enabled":"false"},"Configurations":[]},{"Classification":"spark-env","Properties":{},"Configurations":[{"Classification":"export","Properties":{"SPARK_DAEMON_MEMORY":"4g"},"Configurations":[]}]}]' --scale-down-behavior TERMINATE_AT_TASK_COMPLETION --region us-east-1"""
print(format_command(example))

输出看起来像这样：

aws emr create-cluster \
    --applications Name=spark Name=ganglia Name=hadoop \
    --tags Project=MyProj \
    --ec2-attributes '{
        "ServiceAccessSecurityGroup": "sg-60832c2b", 
        "InstanceProfile": "EMR_EC2_DefaultRole", 
        "EmrManagedMasterSecurityGroup": "sg-b4923dff", 
        "KeyName": "emr-key", 
        "SubnetId": "subnet-3c76ee33", 
        "AdditionalMasterSecurityGroups": [
            "sg-3822994c", 
            "sg-ccc76987"
        ], 
        "AdditionalSlaveSecurityGroups": [
            "sg-3822994c", 
            "sg-ccc76987"
        ], 
        "EmrManagedSlaveSecurityGroup": "sg-dd832c96"
    }' \
    --service-role EMR_DefaultRole \
    --release-label emr-5.14.0 \
    --name Test Cluster \
    --instance-groups '[
        {
            "EbsConfiguration": {
                "EbsBlockDeviceConfigs": [
                    {
                        "VolumeSpecification": {
                            "VolumeType": "gp2", 
                            "SizeInGB": 32
                        }, 
                        "VolumesPerInstance": 1
                    }
                ]
            }, 
            "InstanceCount": 1, 
            "Name": "Master", 
            "InstanceType": "m4.xlarge", 
            "InstanceGroupType": "MASTER"
        }, 
        {
            "EbsConfiguration": {
                "EbsBlockDeviceConfigs": [
                    {
                        "VolumeSpecification": {
                            "VolumeType": "gp2", 
                            "SizeInGB": 32
                        }, 
                        "VolumesPerInstance": 1
                    }
                ]
            }, 
            "InstanceCount": 1, 
            "Name": "CORE", 
            "InstanceType": "m4.xlarge", 
            "InstanceGroupType": "CORE"
        }
    ]' \
    --configurations '[
        {
            "Properties": {
                "spark.eventLog.enabled": "true", 
                "spark.dynamicAllocation.enabled": "false", 
                "spark.sql.avro.compression.codec": "snappy"
            }, 
            "Classification": "spark-defaults", 
            "Configurations": []
        }, 
        {
            "Properties": {}, 
            "Classification": "spark-env", 
            "Configurations": [
                {
                    "Properties": {
                        "SPARK_DAEMON_MEMORY": "4g"
                    }, 
                    "Classification": "export", 
                    "Configurations": []
                }
            ]
        }
    ]' \
    --scale-down-behavior TERMINATE_AT_TASK_COMPLETION \
    --region us-east-1

如何毫不费力地格式化AWS CLI命令

1 个答案: