Question

我以前使用terraform中的以下脚本在bigquery中创建了一个表

resource "google_bigquery_dataset" "my-dataset" {
  dataset_id  = "datasetname"
  description = "description"
}

resource "google_bigquery_table" "mytable" {
  dataset_id = google_bigquery_dataset.my-dataset.dataset_id
  table_id   = "mytable"

  time_partitioning {
    type = "DAY"
  }

  schema = <<EOF
[
  {
      "name": "field_one",
      "type": "STRING",
      "mode": "NULLABLE"
  },
  {
      "name": "field_two",
      "type": "RECORD",
      "mode": "NULLABLE",
      "fields": [
          {
              "name": "sub_field_one",
              "type": "FLOAT",
              "mode": "NULLABLE"
          },
          {
              "name": "sub_field_two",
              "type": "FLOAT",
              "mode": "NULLABLE"
          }
      ]
  },
  {
      "name": "field_three",
      "type": "STRING",
      "mode": "NULLABLE"
  }
]
EOF

}

这很好用，我能够在bigquery中创建mytable。现在，我必须修改该表并向其添加一个新值（field_four。因此制作了此脚本

resource "google_bigquery_dataset" "my-dataset" {
  dataset_id  = "datasetname"
  description = "description"
}

resource "google_bigquery_table" "mytable" {
  dataset_id = google_bigquery_dataset.my-dataset.dataset_id
  table_id   = "mytable"

  time_partitioning {
    type = "DAY"
  }

  schema = <<EOF
[
  {
      "name": "field_one",
      "type": "STRING",
      "mode": "NULLABLE"
  },
  {
      "name": "field_two",
      "type": "RECORD",
      "mode": "NULLABLE",
      "fields": [
          {
              "name": "sub_field_one",
              "type": "FLOAT",
              "mode": "NULLABLE"
          },
          {
              "name": "sub_field_two",
              "type": "FLOAT",
              "mode": "NULLABLE"
          }
      ]
  },
  {
      "name": "field_three",
      "type": "STRING",
      "mode": "NULLABLE"
  },
  {
      "name": "field_four",
      "type": "RECORD",
      "mode": "NULLABLE",
      "fields": [
          {
              "name": "sub_field_three",
              "type": "STRING",
              "mode": "NULLABLE"
          }
      ]
  }
]
EOF

}

当我使用terraform plan运行它时，它表明将添加新字段。但是当我做terraform apply时，出现以下错误

Error: googleapi: Error 400: Cannot convert non partitioned/clustered table to partitioned/clustered table., invalid

  on filename.tf line 10, in resource "google_bigquery_table" "mytable":
  10: resource "google_bigquery_table" "mytable" {

我没有更改表的分区。这里发生了什么。如何将架构更改为

添加新字段？
将字段类型（目前只有null个值）更改为新的内容？

平台详细信息

Terraform v0.12.24
+ provider.google v3.1.0
+ provider.google-beta v3.1.0
+ provider.random v2.2.1

Answer 1

在复制相同的代码之后，我可以毫无问题地添加一个新字段。如评论中所述，当您尝试将非分区表转换为分区表时，会出现以下错误。在这种情况下，问题是由const format = arr => { const res = {}; arr.forEach(item => { if (!res[item.userName]) { res[item.userName] = { totalsteps: 0, list: [] }; } res[item.userName].totalsteps += item.steps; res[item.userName].list.push(item); }); return Object.values(res); }; const resp = { status: 1, UserList: [ { _id: "5e8c04882e31eb0fa0f240dd", userName: "devyansh", starttime: 1586073600, endtime: 1586077200, steps: 1000, createdTime: 1586234504, __v: 0 }, { status: "1", _id: "5e8c04f82e31eb0fa0f240df", userName: "devyansh", starttime: 1586080800, endtime: 1586084400, steps: 47, createdTime: 1586234616, __v: 0 }, { _id: "5e8c282e5040a21c1498d95b", userName: "amit", starttime: 1586170800, endtime: 1586131200, steps: 899, createdTime: 1586243630, __v: 0 } ] }; const result = { status: resp.status, finalList: format(resp.UserList) }; console.log(result);的版本不匹配引起的：用于创建表的版本不支持分区表，因此创建了普通表。在Terraform 0.12.24中运行两个代码时，它都可以工作。

Terraform

关于第二个问题，您无法使用Terraform更改字段的类型。如您所见here，要更改字段的类型，您需要查询作业，这在Terraform中是不可能的，正如您在此open issue中所见。

我希望对您有帮助

模式修改导致分区错误

1 个答案: