我真的是Google Cloud Platform的新生。我正在尝试使用从存储区对象CSV文件提取的数据填充BigQuery表。我创建了一个Python测试脚本来创建和填充表。创建完成,但是当我运行文件时,我陷入了困境。
我的脚本:
from google.cloud import bigquery
from google.cloud.bigquery import LoadJobConfig
from google.cloud.bigquery import SchemaField
client = bigquery.Client()
dataset_ref = client.dataset('datasetname')
## Create the table
schema = [
bigquery.SchemaField('start_date', 'DATETIME', mode='NULLABLE'),
bigquery.SchemaField('start_station_code', 'INTEGER', mode='NULLABLE'),
bigquery.SchemaField('end_date', 'DATETIME', mode='NULLABLE'),
bigquery.SchemaField('end_station_code', 'INTEGER', mode='NULLABLE'),
bigquery.SchemaField('duration_sec', 'INTEGER', mode='NULLABLE'),
bigquery.SchemaField('is_member', 'INTEGER', mode='NULLABLE')
]
table_ref = dataset_ref.table('tablename')
table = bigquery.Table(table_ref, schema=schema)
table = client.create_table(table) # API request
## Loading data
SCHEMA = [
SchemaField('start_date', 'DATETIME', mode='NULLABLE'),
SchemaField('start_station_code', 'INTEGER', mode='NULLABLE'),
SchemaField('end_date', 'DATETIME', mode='NULLABLE'),
SchemaField('end_station_code', 'INTEGER', mode='NULLABLE'),
SchemaField('duration_sec', 'INTEGER', mode='NULLABLE'),
SchemaField('is_member', 'INTEGER', mode='NULLABLE')
]
#table_ref = client.dataset('dataset_name').table('table_name')
load_config = LoadJobConfig()
load_config.skip_leading_rows = 1
load_config.schema = SCHEMA
uri = 'gs://gcp-development/object.csv'
load_job = client.load_table_from_uri(
uri,
table_ref,
job_config=load_config)
load_job.result()
destination_table = client.get_table(table_ref)
print('Loaded {} rows.'.format(destination_table.num_rows))
按照documentation,这似乎是正确的。但是,我遇到了我不理解的以下错误,并且我不知道如何查看日志以获取更多详细信息。
错误:
google.api_core.exceptions.BadRequest: 400 Error while reading data, error message: CSV table encountered too many errors, giving up. Rows: 1; errors: 1. Please look into the error stream for more details.
错误流在哪里?我尝试过:
GET https://www.googleapis.com/bigquery/v2/projects/projectId/queries/jobId
在troubleshooting documentation之后,但我什么也没找到。
感谢您的帮助。