Question

我有一个云功能，目前将采用登陆云存储的.csv文件，并将该文件加载到大查询表中。问题是它附加到它，我需要它覆盖，我找到了一种方法来使用命令行--replace但不确定如何使用云函数在.json中执行此操作。以下是我目前的代码：

exports.ToBigQuery_Stage = (event, callback) => {
  const file = event.data;
  const context = event.context;

  const BigQuery = require('@google-cloud/bigquery');
  const Storage = require('@google-cloud/storage');

  const projectId = "gas-ddr";
  const datasetId = "gas_ddr_qc_stage";
  const bucketName = file.bucket;
  const filename = file.name;



  // Do not use the ftp_files Bucket to ensure that the bucket does not get crowded. 
  // Change bucket to gas_ddr_files_staging
  // Set the table name (TableId) to the full file name including date, 
  // this will give each table a new distinct name and we can keep a record of all of the files recieved.
  // This may not be the best way to do this... at some point we will need to archive and delete prior records. 
    const dashOffset = filename.indexOf('-');
  const tableId = filename.substring(0, dashOffset) + "_STAGE";


  console.log(`Load ${filename} into ${tableId}.`);

 // Instantiates clients
  const bigquery = new BigQuery({
    projectId: projectId,
  });

  const storage = Storage({
    projectId: projectId,
  });

  const metadata = {
  allowJaggedRows: true,
  skipLeadingRows: 1

 };



  let job;

  // Loads data from a Google Cloud Storage file into the table
  bigquery
    .dataset(datasetId)
    .table(tableId)
    .load(storage.bucket(bucketName).file(filename),metadata)
    .then(results => {
      job = results[0];
      console.log(`Job ${job.id} started.`);

      // Wait for the job to finish
      return job;
    })
    .then(metadata => {
      // Check the job's status for errors
      const errors = metadata.status.errors;
      if (errors && errors.length > 0) {
        throw errors;
      }
    })
    .then(() => {
      console.log(`Job ${job.id} completed.`);
    })
    .catch(err => {
      console.error('ERROR:', err);
    });

  callback();
};

Answer 1

您可以将其添加到metadata：

const metadata = {
allowJaggedRows: true,
skipLeadingRows: 1,
writeDisposition: 'WRITE_TRUNCATE'
};

您可以在documentation中找到更多信息。

使用云函数将数据加载到Big Query Table中，它附加到表中，我需要它来替换

1 个答案: