Question

我创建了一个连续三列的csv文件。在google bigquery创建了一个带有csv文件的表的数据集....为此我完成了我的java代码...但是现在我必须添加一个在java代码中动态存在的新列。？

// Main method for data print.
@SuppressWarnings({ "rawtypes", "unchecked" })
public static void main(String[] args) throws IOException, InterruptedException {

    // Create a new BigQuery client authorized via OAuth 2.0 protocol
    Bigquery bigquery = createAuthorizedClient();
    TableRow row = new TableRow();
    row.set("Column1", "Sample23");
    row.set("Column2", 7);
    row.set("Column3", "Sample25");
    TableDataInsertAllRequest.Rows rows = new TableDataInsertAllRequest.Rows();
    rows.setJson(row);
    List rowList = new ArrayList();
    rowList.add(rows);
    TableDataInsertAllRequest content = 
        new TableDataInsertAllRequest().setRows(rowList);
    TableDataInsertAllResponse response = bigquery.tabledata().insertAll(PROJECT_ID, DATASET_ID, TABLE_ID, content).execute();
    System.out.println("Final response = " + response);

}

Answer 1

有两个表操作Update和Patch。

您需要使用更新命令将新列添加到架构中。

重要的附注：

顺序很重要。如果更改排序，它将看起来像一个不兼容的架构。
您只能在表格末尾添加新字段。在旧列上，您可以选择将所需更改为可为空。
您无法向现有架构添加必填字段。
您无法删除旧字段，一旦指定了表格的架构，您就无法在不先删除与其关联的所有数据的情况下进行更改。如果要更改表的模式，则必须指定WRITE_TRUNCATE的writeDisposition。有关更多信息，请参阅作业资源。

Here is an example卷曲会话，它将字段添加到架构中。适应Java应该相对容易。它使用来自here

使用Table.Update（）时，必须再次包含完整的表架构。如果您未提供完全匹配的架构，则可以获得：Provided Schema does not match Table。例如，我没有注意细节，在我的一个更新电话中，我没有包含像created这样的旧字段，但它失败了。

Answer 2

实际上我没有在我的java代码中使用任何作业。我只是创建了一个数据集，其中一个表包含三列中的行。现在我必须在java代码中添加新列而不是在csv文件中。我发布了完整的源代码：

public class BigQueryJavaGettingStarted {


    // Define required variables.
    private static final String PROJECT_ID = "nvjsnsb";
    private static final String DATASET_ID = "nvjjvv";
    private static final String TABLE_ID = "sampledata";
    private static final String CLIENTSECRETS_LOCATION = "client_secrets.json";
    static GoogleClientSecrets clientSecrets = loadClientSecrets(CLIENTSECRETS_LOCATION);
    // Static variables for API scope, callback URI, and HTTP/JSON functions
    private static final List<String> SCOPES = Arrays.asList(BigqueryScopes.BIGQUERY);
    private static final String REDIRECT_URI = "https://www.example.com/oauth2callback";
    // Global instances of HTTP transport and JSON factory objects.
    private static final HttpTransport TRANSPORT = new NetHttpTransport();
    private static final JsonFactory JSON_FACTORY = new JacksonFactory();
    private static GoogleAuthorizationCodeFlow flow = null;


    // Main method for data print.
    @SuppressWarnings({ "rawtypes", "unchecked" })
        public static void main(String[] args) throws IOException, InterruptedException {

            // Create a new BigQuery client authorized via OAuth 2.0 protocol
            Bigquery bigquery = createAuthorizedClient();
            TableRow row = new TableRow();
            row.set("Column1", "OneMoreCol1");
            row.set("Column2", 79);
            row.set("Column3", "OneMoreCol2");
            TableDataInsertAllRequest.Rows rows = new TableDataInsertAllRequest.Rows();
            rows.setJson(row);
            List rowList = new ArrayList();
            rowList.add(rows);
            TableDataInsertAllRequest content = new TableDataInsertAllRequest().setRows(rowList);
            TableDataInsertAllResponse response = bigquery.tabledata().insertAll(PROJECT_ID, DATASET_ID, TABLE_ID, content).execute();
            System.out.println("Final response = " + response);

        }


    // Create Authorized client.
    public static Bigquery createAuthorizedClient() throws IOException {

        String authorizeUrl = new GoogleAuthorizationCodeRequestUrl(
                clientSecrets,
                REDIRECT_URI,
                SCOPES).setState("").build();

        System.out.println("Paste this URL into a web browser to authorize BigQuery Access:\n" + authorizeUrl);

        System.out.println("... and type the code you received here: ");
        BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
        String authorizationCode = in.readLine();

        // Exchange the auth code for an access token and refresh token
        Credential credential = exchangeCode(authorizationCode);

        return new Bigquery(TRANSPORT, JSON_FACTORY, credential);

    }


    // Exchange code method.
    static Credential exchangeCode(String authorizationCode) throws IOException {

        GoogleAuthorizationCodeFlow flow = getFlow();
        GoogleTokenResponse response =
            flow.newTokenRequest(authorizationCode).setRedirectUri(REDIRECT_URI).execute();
        return flow.createAndStoreCredential(response, null);

    }


    // Get flow.
    static GoogleAuthorizationCodeFlow getFlow() {

        if (flow == null) {
            HttpTransport httpTransport = new NetHttpTransport();
            JacksonFactory jsonFactory = new JacksonFactory();
            flow = new GoogleAuthorizationCodeFlow.Builder(httpTransport,
                    jsonFactory,
                    clientSecrets,
                    SCOPES)
                .setAccessType("offline").setApprovalPrompt("force").build();
        }
        return flow;

    }


    // Load client secrets.
    private static GoogleClientSecrets loadClientSecrets(String clientSecretsLocation) {
        try {
            clientSecrets = GoogleClientSecrets.load(new JacksonFactory(),
                    new InputStreamReader(BigQueryJavaGettingStarted.class.getResourceAsStream(clientSecretsLocation)));
        } catch (Exception e) {
            System.out.println("Could not load client_secrets.json");
            e.printStackTrace();
        }
        return clientSecrets;

    }
}

如何动态地将新列添加到bigquery中已存在的表中？

2 个答案: