Question

我正在尝试通过python客户端在Bigquery创建一个表。文档使用bigquery.SchemaField('name', 'TYPE')来定义字段。虽然它似乎不适用于 ARRAYS 或 STRUCTS 。这是我试图创建的STRULTS字段阵列：

bigquery.SchemaField('owners', 'ARRAY<STRUCT<emailAddress STRING, displayName STRING>>', 'REPEATABLE'),

如果我使用上面的字段定义，我会收到以下API错误：

400 POST https://www.googleapis.com/bigquery/v2/projects/import-sheet/datasets/sheetgo/tables: Invalid value for: ARRAY<STRUCT<emailAddress STRING, displayName STRING>> is not a valid value

整个代码：

schema = [
            bigquery.SchemaField('user', 'STRING'),
            bigquery.SchemaField('id', 'STRING'),
            bigquery.SchemaField('service_origin', 'STRING'),
            bigquery.SchemaField('name', 'STRING'),
            bigquery.SchemaField('mimeType', 'STRING'),
            bigquery.SchemaField('createdAt', 'DATETIME'),
            bigquery.SchemaField('ownedByMe', 'BOOLEAN'),
            bigquery.SchemaField('owners', 'ARRAY<STRUCT<emailAddress STRING, displayName STRING>>', 'REPEATABLE'),
            bigquery.SchemaField('parents', 'ARRAY<STRING>', 'REPEATABLE'),
            bigquery.SchemaField('teamDriveId', 'STRING'),
            bigquery.SchemaField('permissions', 'STRING'),
            bigquery.SchemaField('shared', 'BOOLEAN'),
            bigquery.SchemaField('writersCanShare', 'BOOLEAN'),
            bigquery.SchemaField('sharingUser', 'STRING'),
            bigquery.SchemaField('version', 'STRING'),
            bigquery.SchemaField('size', 'FLOAT'),
            bigquery.SchemaField('data_properties', 'ARRAY<STRUCT<'
                                                        'rows INTEGER,'
                                                        'cells_with_importrange ARRAY<'
                                                                                    'STRUCT<'
                                                                                        'row_index INTEGER,'
                                                                                        'col_index INTEGER,'
                                                                                        'importrange STRING'
                                                                                      '>'
                                                                                    '>,'
                                                        'tab_name STRING,'
                                                        'cell_count FLOAT,'
                                                        'header_rows ARRAY<STRING>,'
                                                    '>>', 'REPEATABLE'),
            bigquery.SchemaField('timezone', 'STRING'),
            bigquery.SchemaField('locale', 'STRING'),
            bigquery.SchemaField('last_scansheet', 'STRING'),
        ]

        bigquery_client = bigquery.Client(PROJECT_ID)
        dataset_ref = bigquery_client.dataset("eita")

        table_ref = dataset_ref.table(table_id)
        table = bigquery.Table(table_ref, schema=schema)

        table = bigquery_client.create_table(table)

更新

感谢Willian Fuks，我得到了这个工作。架构的最终结果如下所示：

schema = [
            bigquery.SchemaField('user', 'STRING'),
            bigquery.SchemaField('id', 'STRING'),
            bigquery.SchemaField('service_origin', 'STRING'),
            bigquery.SchemaField('name', 'STRING'),
            bigquery.SchemaField('mimeType', 'STRING'),
            bigquery.SchemaField('createdAt', 'DATETIME'),
            bigquery.SchemaField('ownedByMe', 'BOOLEAN'),
            bigquery.SchemaField('owners', 'RECORD', mode='REPEATED',
                fields=(
                    bigquery.SchemaField('emailAddress', 'STRING'),
                    bigquery.SchemaField('displayName', 'STRING')
                )
            ),
            bigquery.SchemaField('parents', 'STRING', mode='REPEATED'),
            bigquery.SchemaField('teamDriveId', 'STRING'),
            bigquery.SchemaField('permissions', 'STRING'),
            bigquery.SchemaField('shared', 'BOOLEAN'),
            bigquery.SchemaField('writersCanShare', 'BOOLEAN'),
            bigquery.SchemaField('sharingUser', 'STRING'),
            bigquery.SchemaField('version', 'STRING'),
            bigquery.SchemaField('size', 'FLOAT'),
            bigquery.SchemaField('data_properties', 'RECORD', mode='REPEATED',
                 fields=(
                     bigquery.SchemaField('rows', 'INTEGER'),
                     bigquery.SchemaField('cells_with_importrange', 'RECORD', mode='REPEATED',
                          fields=(
                              bigquery.SchemaField('row_index', 'INTEGER'),
                              bigquery.SchemaField('col_index', 'INTEGER'),
                              bigquery.SchemaField('importrange', 'STRING'),
                          )
                      ),
                     bigquery.SchemaField('tab_name', 'STRING'),
                     bigquery.SchemaField('cell_count', 'FLOAT'),
                     bigquery.SchemaField('header_rows', 'STRING', mode='REPEATED')
                 )
             ),
            bigquery.SchemaField('timezone', 'STRING'),
            bigquery.SchemaField('locale', 'STRING'),
            bigquery.SchemaField('last_scansheet', 'STRING'),
        ]

Answer 1

SchemaField的构造函数合同与您使用的合同有expect different inputs。

请改为尝试：

schema = [
    (...),
    SchemaField('owners', 'RECORD', mode='REPEATED',
        fields=(SchemaField('emailAddress', 'STRING'),
                SchemaField('displayName', 'STRING')
        )
    ),
    (...)
]

主要思想是使用其他SchemaField定义来定义记录字段内的字段。

Answer 2

如果要使用标准SQL类型名称而不是使用旧SQL类型和SchemaField，则可以执行查询来创建表：

CREATE TABLE dataset.table_name
(
  user STRING,
  id STRING,
  service_origin STRING,
  name STRING,
  mimeType STRING,
  createdAt DATETIME,
  ownedByMe BOOL,
  owners ARRAY<STRUCT<emailAddress STRING, displayName STRING>>,
  parents ARRAY<STRING>,
  teamDriveId STRING,
  permissions STRING,
  shared BOOL,
  writersCanShare BOOL,
  sharingUser STRING,
  version STRING,
  size FLOAT64,
  data_properties
    ARRAY<STRUCT<`rows` INT64,
                 cells_with_importrange ARRAY<STRUCT<row_index INT64, col_index INT64, importrange STRING>>,
                 tab_name STRING, cell_count FLOAT64, header_rows ARRAY<STRING>>>,
  timezone STRING,
  locale STRING,
  last_scansheet STRING
);

Bigquery python SchemaField（），ARRAY为STRUCTS

整个代码：

更新

2 个答案: