Question

我正在使用terraform，但没有获得正确的参数来创建我的胶水作业。由于我不是Terraform专业人士（我开始），所以我想知道它的工作原理。

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/glue_job#glue_version

我对使用这些参数的胶水作业资源没有很好的更新：

resource "aws_glue_job" "job_name" {
  name     = "job_name"
  description  = "job-desc"
  role_arn = "${aws_iam_role.service-name.arn}"
  max_capacity  = 2
  max_retries  = 1
  timeout      = 60

  command {
    script_location = "s3://my_bucket"
    python_version  = "3"
  }

  default_arguments = {    
    "--job-language"    = "python"
    "--ENV"             = "env"
    "--spark-event-logs-path" = "s3://my_bucket"
    "--job-bookmark-option" = "job-bookmark-enable"
    "--glue_version" = "2.0"
    "--worker_type" = "G.1X"
    "--enable-spark-ui" = "true"
  }

  execution_property {
    max_concurrent_runs = 1
  }
}

我不知道这些参数在哪里以及如何放置。你能帮我吗？

    "--glue_version" = "2.0"
    "--worker_type" = "G.1X"

致谢。

Answer 1

glue_version和worker_type自变量与default_arguments块位于同一级别，而不是在其内部。

将它们移出后，您的resource块可能看起来像这样：

resource "aws_glue_job" "job_name" {
  name         = "job_name"
  description  = "job-desc"
  role_arn     = "${aws_iam_role.service-name.arn}"
  max_capacity = 2
  max_retries  = 1
  timeout      = 60
  glue_version = "2.0"
  worker_type  = "G.1X"

  command {
    script_location = "s3://my_bucket"
    python_version  = "3"
  }

  default_arguments = {    
    "--job-language"          = "python"
    "--ENV"                   = "env"
    "--spark-event-logs-path" = "s3://my_bucket"
    "--job-bookmark-option"   = "job-bookmark-enable"
    "--enable-spark-ui"       = "true"
  }

  execution_property {
    max_concurrent_runs = 1
  }
}

编辑

您使用的版本2.30.0不支持aws_glue_job资源的这些参数。

直到2.34.0版本的AWS Provider才添加glue_version自变量。

worker_type参数直到版本2.39.0才添加。

您将需要更新提供程序以支持这些参数。

Terraform胶合作业无法正确创建

1 个答案: