在BigQuery和c#中使用模式自动检测

时间:2017-07-03 15:38:53

标签: c# csv google-bigquery

目标:

使用Google.Cloud.BigQuery.V2 -Pre c#包将.csv文件上传到Google BigQuery,如果它不存在则自动创建一个表。

问题:

possible自动检测数据库方案并创建一个新表(如果它不存在)。默认情况下启用此option。但是,如果我尝试将.csv上载到不存在的表,则不会创建新表。它向我显示了一个错误信息。

我认为它可能与TableSchema有关,因为我不知道如何基于.csv文件创建TableSchema。

Google.GoogleApiException: 'Google.Apis.Requests.RequestError

Not found: Table project-id-here:dataset-id-here.table-id-here [404]

Errors [
Message[Not found: Table project-id-here:dataset-id-here.table-id-here] Location[ - ] Reason[notFound] Domain[global]
]

代码:

    string fileName = @"C:\result.csv";
    string projectId = "project-id-here";
    string datasetId = "dataset-id-here";
    string tableId = "table-id-here";

    BigQueryClient client = BigQueryClient.Create(projectId);
    TableSchema schema = null;

    using (FileStream stream = File.Open(fileName, FileMode.Open))
    {

        Console.WriteLine("Starting polling...");
        // This example uploads data to an existing table. If the upload will create a new table
        // or if the schema in the JSON isn't identical to the schema in the table,
        // create a schema to pass into the call instead of passing in a null value.
         BigQueryJob job = client.UploadCsv(datasetId, tableId, schema, stream, new UploadCsvOptions { SkipLeadingRows = 10 });

        // Use the job to find out when the data has finished being inserted into the table,
        // report errors etc.

        BigQueryJob result = job.PollUntilCompleted();
        // If there are any errors, display them *then* fail.
        if (result.Status.ErrorResult != null)
        {
            foreach (ErrorProto error in result.Status.Errors)
            {
                Console.WriteLine(error.Message);
            }
        }

        Console.WriteLine("Polling ended");

    }

提前致谢。 :)

1 个答案:

答案 0 :(得分:0)

此功能已在1.0.0-beta13版本中添加。创建表需要自动模式,或者您必须手动指定。数据集必须在那里,当我尝试上传我的数据时它不存在。

https://github.com/GoogleCloudPlatform/google-cloud-dotnet/issues/1166