插入从选择查询中检索的MySQL结果集？

Question

我在服务器x 上有一个MySQL数据库，在服务器y 上有另一个数据库。

我试图从位于服务器x上的名为 testx 的表中获取记录，并将INSERT这些记录放入名为 testy 。所以我所做的是执行SELECT语句并将其存储到resultset中。然后，我尝试在INSERT resultset循环中迭代while语句。这是我的示例代码：

private static void cloneTableAndAlter() throws ClassNotFoundException, SQLException, InstantiationException, IllegalAccessException, InterruptedException {
        Connection connForSource = getConnectionForSource();
        Connection connForTarget = getConnectionForTarget();

        if (connForSource != null && connForTarget != null) {
            try {
                Statement st = connForSource.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY,
                        java.sql.ResultSet.CONCUR_READ_ONLY);

                String selectStatement = "SELECT field1, field2 FROM dbx.testx where time between ('2016-09-01 00:00:00') and ('2016-09-03 23:59:59');";

                String insertStatement = "INSERT INTO testy(" + "field1," + "field2)" + 
                        "VALUES(?,?) " +
                        "ON DUPLICATE KEY UPDATE field1 = VALUES(field1);"; <----field1 is a unique key but not primary

                st.setFetchSize(Integer.MIN_VALUE);

                ResultSet resultSetForSelect = st.executeQuery(selectStatement);
                PreparedStatement preparedStatement = connForTarget.prepareStatement(insertStatement);
                int count = 0;
                while (resultSetForSelect.next()) {
                    ++count;
                    TableDetails tableDetails = setTableDetails(resultSetForSelect); <--- i'm getting the value from the resultset and setting it to my DTO class.
                    getTableDetails(preparedStatement, tableDetails); <--- setting the values back during the insert from the DTO
                    preparedStatement.executeUpdate();
                    System.out.println(count + "rows affected");
                }
            } catch (SQLException e) {
                e.printStackTrace();
            }
        }
    }

注意：源表（ testx ）的记录超过一百万。

我能够插入，但后来我感觉插入有点慢，我每秒插入45-50次插入。我哪里会迟到？

有什么方法可以优化此操作并增加插入，或者这是插入大型数据集的本质？

任何帮助都可以得到赞赏。

Answer 1

你可以通过其他方式做到：

使用此Concats创建选择

select GROUP_CONCAT(
    CONCAT(" ('",field1,"','",field2,"')")) as vals
from my_table;

结果如下所示：

<强>结果

mysql> select GROUP_CONCAT(
    -> CONCAT(" ('",field1,"','",field2,"')")) as vals
    -> from my_table;
+---------------------------------------------------------------------------------------------------------+
| vals                                                                                                    |
+---------------------------------------------------------------------------------------------------------+
|  ('O1','AC'), ('O1','PT'), ('O2','PT'), ('O3','MI'), ('O3','PT'), ('O4','EG'), ('O4','PT'), ('O5','PT') |
+---------------------------------------------------------------------------------------------------------+
1 row in set (0,00 sec)

mysql>

并且您可以在insert语句中指向concat，并且只读取一行并且只写入并执行一个语句。

您的代码

    String selectStatement = "SELECT GROUP_CONCAT(CONCAT(" ('",field1,"','",field2,"')")) as vals
         FROM dbx.testx where time between ('2016-09-01 00:00:00') and ('2016-09-03 23:59:59');";

    ResultSet resultSetForSelect = st.executeQuery(selectStatement);


    String insertStatement = "INSERT INTO testy(" + "field1," + "field2)" + 
                            + String from result +
                            "ON DUPLICATE KEY UPDATE field1 = VALUES(field1);"; <----field1 is a unique key but not primary


# execute one time