Question

有没有办法从Google Cloud存储上传数据到Google BigQuery而无需通过.NET代码下载？就像它可以在UI上完成一样。

以下是文档中的示例： https://cloud.google.com/bigquery/docs/loading-data-cloud-storage#bigquery-import-gcs-file-csharp

    StorageClient gcsClient = StorageClient.Create();

    using (var stream = new MemoryStream())
    {
        // Set Cloud Storage Bucket name. This uses a bucket named the same as the project.
        string bucket = projectId;
        // If folder is passed in, add it to Cloud Storage File Path using "/" character
        string filePath = string.IsNullOrEmpty(folder) ? fileName : folder + "/" + fileName;
        // Download Google Cloud Storage object into stream
        gcsClient.DownloadObject(projectId, filePath, stream);

        // This example uploads data to an existing table. If the upload will create a new table
        // or if the schema in the JSON isn't identical to the schema in the table,
        // create a schema to pass into the call instead of passing in a null value.
        BigQueryJob job = client.UploadJson(datasetId, tableId, null, stream);
        // Use the job to find out when the data has finished being inserted into the table,
        // report errors etc.

        // Wait for the job to complete.
        job.PollUntilCompleted();
    }

Answer 1

这里的方式

public void ImportDataFromCloudStorage(BigQueryClient client, string uri, TableReference reference, WriteDisposition disposition = WriteDisposition.WriteIfEmpty)
    {
      BigQueryJob job = client.CreateLoadJob(uri, reference, null, new CreateLoadJobOptions()
      {
        FieldDelimiter = ";",
        SkipLeadingRows = 1,
        AllowQuotedNewlines = true,
        WriteDisposition = disposition
      });

      job.PollUntilCompleted();
    }

Google BigQuery - 直接从Google云端存储上传数据

1 个答案: