Question

这应该用CSV读取然后写入bigquery。但是，当它运行时，没有写入任何内容，并且没有记录错误。我读到我需要编写一个csv，然后把它变成一个Octet Stream。我不确定这是否与google bigquery兼容。

function test(){
try{
  var tableReference = BigQuery.newTableReference();
  tableReference.setProjectId(PROJECT_ID);
  tableReference.setDatasetId(datasetId);
  tableReference.setTableId(tableId);
  var schema = "CUSTOMER:string, CLASSNUM:integer, CLASSDESC:string, CSR:string, CSR2:string, INSURANCE:string, REFERRALGENERAL:string, REFERRALSPECIFIC:string, NOTES:string, INMIN:integer, INHR:integer, OUTMIN:integer, OUTHR:integer, WAITMIN:integer, WAITHR:integer, DATETIMESTAMP:float, DATEYR:integer,DATEMONTH:integer, DATEDAY:integer";
  var load = BigQuery.newJobConfigurationLoad();
  load.setDestinationTable(tableReference);
  load.setSourceUris(URIs);
  load.setSourceFormat('NEWLINE_DELIMITED_JSON');
  load.setSchema(schema);
  load.setMaxBadRecords(0);
  load.setWriteDisposition('WRITE_TRUNCATE');

  var configuration = BigQuery.newJobConfiguration();
  configuration.setLoad(load);

  var newJob = BigQuery.newJob();
  newJob.setConfiguration(configuration);


  var loadr = DriveApp.getFilesByName("test.csv");
  var x = loadr.next().getBlob();
  Logger.log(x.getDataAsString());

  var d = DriveApp.getFilesByName("test.csv");
  var id = d.next().getId();
  Logger.log(id);
  var data = DocsList.getFileById(id).getBlob().getDataAsString();
  var mediaData = Utilities.newBlob(data, 'application/octet-stream');

  BigQuery.Jobs.insert(newJob, PROJECT_ID, mediaData) 
 }
 catch(error){Logger.log("A" + error.message);}
}

Answer 1

您的sourceFormat对于CSV文件是错误的：

数据文件的格式。对于CSV文件，请指定“CSV”。对于数据存储备份，指定“DATASTORE_BACKUP”。用于换行符分隔 JSON，指定“NEWLINE_DELIMITED_JSON”。默认值为CSV。

https://developers.google.com/bigquery/docs/reference/v2/jobs#configuration.load.sourceUris

另一方面，我认为您完全不需要load.setSourceUris(URIs);，因为您尝试从本地文件加载，而不是从Google云端存储加载。检查这个python示例https://developers.google.com/bigquery/loading-data-into-bigquery

Apps脚本写入Big Query未知错误

1 个答案: