Question

Cloudformation中是否有类似于 terraform modules 的内容，您可以在其中创建参数化模板（资源组，而不是 CF模板），然后在< em> CF模板多次使用不同的参数？

我在 CF模板中看到了很多bolierplate YAML，我正在寻找一种重构它的方法。我之前使用过terraform，它提供了这种功能。

示例：

我正在创建许多AWS Glue作业，其中许多作业只有2个参数不同，但每个定义都是25行代码。

Resources:
  myGlueJob1:
    Type: AWS::Glue::Job
    Properties:
      ExecutionProperty:
        MaxConcurrentRuns: 1
      MaxRetries: 3
      Name: myGlueJob1
      Role: arn:aws:iam::xxxxxxx:role/XXXXXXXXXXXXXX
      Command:
        Name: glueetl
        ScriptLocation: XXXXXX
      DefaultArguments:
        "--ga_project_id": PARAM1-THAT-DIFFERS
        "--ga_view_id": PARAM2-THAT-DIFFERS
        "--ga_service_account_keyfile": gc.key-SAME_FOR_ALL_RESOURCES
        "--date": YESTERDAY-SAME_FOR_ALL_RESOURCES
        "--temp_gcs_bucket": "foobar-SAME_FOR_ALL_RESOURCES"
        "--output_path": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
        "--job-language": scala
        "--class": GlueApp
        "--TempDir": "s3://some-other-s3-path-SAME_FOR_ALL_RESOURCES"
        "--extra-files": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
        "--extra-jars": "s3://looooooooooooooooooooooooots-of-s3-paths-SAME_FOR_ALL_RESOURCES"
  myGlueJob2:
    Type: AWS::Glue::Job
    Properties:
      ExecutionProperty:
        MaxConcurrentRuns: 1
      MaxRetries: 3
      Name: myGlueJob2
      Role: arn:aws:iam::xxxxxxx:role/XXXXXXXXXXXXXX
      Command:
        Name: glueetl
        ScriptLocation: XXXXXX
      DefaultArguments:
        "--ga_project_id": PARAM1-THAT-DIFFERS
        "--ga_view_id": PARAM2-THAT-DIFFERS
        "--ga_service_account_keyfile": gc.key-SAME_FOR_ALL_RESOURCES
        "--date": YESTERDAY-SAME_FOR_ALL_RESOURCES
        "--temp_gcs_bucket": "foobar-SAME_FOR_ALL_RESOURCES"
        "--output_path": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
        "--job-language": scala
        "--class": GlueApp
        "--TempDir": "s3://some-other-s3-path-SAME_FOR_ALL_RESOURCES"
        "--extra-files": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
        "--extra-jars": "s3://looooooooooooooooooooooooots-of-s3-paths-SAME_FOR_ALL_RESOURCES"

我可以想象一个看起来像这样的解决方案：

Module:
    Type: Me::MyGlueJob
    Resouces:
        Type: AWS::Glue::Job
        Properties:
          ExecutionProperty:
            MaxConcurrentRuns: 1
          MaxRetries: 3
          Name: myGlueJob2
          Role: arn:aws:iam::xxxxxxx:role/XXXXXXXXXXXXXX
          Command:
            Name: glueetl
            ScriptLocation: XXXXXX
          DefaultArguments:
            "--ga_project_id": {{ MY_PARAM1 }}
            "--ga_view_id": {{ MY_PARAM2 }}
            "--ga_service_account_keyfile": gc.key-SAME_FOR_ALL_RESOURCES
            "--date": YESTERDAY-SAME_FOR_ALL_RESOURCES
            "--temp_gcs_bucket": "foobar-SAME_FOR_ALL_RESOURCES"
            "--output_path": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
            "--job-language": scala
            "--class": GlueApp
            "--TempDir": "s3://some-other-s3-path-SAME_FOR_ALL_RESOURCES"
            "--extra-files": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
            "--extra-jars": "s3://looooooooooooooooooooooooots-of-s3-paths-SAME_FOR_ALL_RESOURCES"
Resources:
  myGlueJob1:
    Type: Me::MyGlueJob
    Properties:
        MY_PARAM1: value-for-job1
        MY_PARAM2: value-for-job1
  myGlueJob2:
    Type: Me::MyGlueJob
    Properties:
        MY_PARAM1: value-for-job2
        MY_PARAM2: value-for-job2

非常感谢任何关于最佳做法的提示。

Answer 1

您应该可以使用jinja2 templates to generate actual CloudFormation模板。

在你的情况下它应该是这样的：

{% set job_params = [
  ["value-for-job1", "value-for-job1"],
  ["value-for-job2", "value-for-job2"]
] %}

Resources:
{% for params in job_params %}
  myGlueJob{{loop.index}}:
    Type: AWS::Glue::Job
    Properties:
      ExecutionProperty:
        MaxConcurrentRuns: 1
      MaxRetries: 3
      Name: myGlueJob1
      Role: arn:aws:iam::xxxxxxx:role/XXXXXXXXXXXXXX
      Command:
        Name: glueetl
        ScriptLocation: XXXXXX
      DefaultArguments:
        "--ga_project_id": "{{params[0]}}"
        "--ga_view_id": "{{params[1]}}"
        "--ga_service_account_keyfile": gc.key-SAME_FOR_ALL_RESOURCES
        "--date": YESTERDAY-SAME_FOR_ALL_RESOURCES
        "--temp_gcs_bucket": "foobar-SAME_FOR_ALL_RESOURCES"
        "--output_path": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
        "--job-language": scala
        "--class": GlueApp
        "--TempDir": "s3://some-other-s3-path-SAME_FOR_ALL_RESOURCES"
        "--extra-files": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
        "--extra-jars": "s3://looooooooooooooooooooooooots-of-s3-paths-SAME_FOR_ALL_RESOURCES"
{% endfor %}

Cloudformation自定义模块

1 个答案: