Question

我正在尝试使用Java准备的语句将数据插入分区的impala表中。插入在非分区上运行良好，但分区表出现错误。

我需要检查分区与非分区的性能差异，我觉得最好是对连接使用准备好的语句。


create_query = "CREATE TABLE IF NOT EXISTS tbl_raw_flow_v9 (source_ip string,destination_ip string,protocol_number int,) partitioned by (year int) stored as parquet";

    private static final String sqlCompiledQuery = "INSERT INTO tbl_raw_flow_v9 (source_ip,destination_ip,protocol_number" +
            "PARTITION (year) VALUES (?,?,?,?)";


    private static Connection connectViaDS() throws Exception {
        Connection connection = null;

        Class.forName("com.cloudera.impala.jdbc41.Driver");

        connection = DriverManager.getConnection(CONNECTION_URL);

        return connection;

    }


# my try block

try {
            connection = connectViaDS();

            connection.setAutoCommit(false);

            preparedStatement = connection.prepareStatement(sqlCompiledQuery);

            while(counter <total) {

                for (int index = 1; index <= records; index++) {

                    counter++;

                    preparedStatement.setString(1, "s_ip" + String.valueOf(index));

                    preparedStatement.setString(2, "d_ip" + String.valueOf(index));

                    preparedStatement.setInt(3, protocol_no + index);

                    preparedStatement.setInt(4, yr + flag);

                    preparedStatement.addBatch();

                }
                preparedStatement.executeBatch();

                preparedStatement.clearBatch();

                connection.commit();

                System.out.println("Counter = "+counter);

                flag++;


            }

        }

我不知道我在使用准备好的语句时是否犯了错误或还有其他问题。我可以使用

将值插入分区表

"INSERT INTO tbl_raw_flow_v9 (source_ip,destination_ip,protocol_number) PARTITION (year) VALUES ("s_ip","d_ip",234,2010);

当我使用预付费语句时，在java-

中出现以下错误

java.sql.SQLException: [Simba][JDBC](11420) Error, parameter metadata not populated.
    at com.cloudera.exceptions.ExceptionConverter.toSQLException(Unknown Source)
    at com.cloudera.jdbc.common.SPreparedStatement.<init>(Unknown Source)
    at com.cloudera.jdbc.common4.C4SPreparedStatement.<init>(Unknown Source)
    at com.cloudera.jdbc.jdbc41.S41PreparedStatement.<init>(Unknown Source)
    at com.cloudera.jdbc.jdbc41.JDBC41ObjectFactory.createPreparedStatement(Unknown Source)
    at com.cloudera.impala.jdbc41.ImpalaJDBC41ObjectFactory.createPreparedStatement(Unknown Source)
    at com.cloudera.jdbc.common.SConnection.prepareStatement(Unknown Source)
    at impala_test.writeInABatchWithCompiledQuery(impala_test.java:100)
    at impala_test.main(impala_test.java:52)

如果有人可以对此有所帮助，将会非常有帮助。

无法使用准备好的语句在分区表中插入数据

0 个答案: