Question

我将数据访问代码写入批处理操作，以提高我的应用程序的性能。

我目前正在删除，但这很容易适用于插入和更新。

我有5张表要处理。这是我尝试的两种方式（辅助函数位于底部）：

public void deleteFooList(final int[] fooIds)
        throws SQLException {

    Connection connection = dataSource.getConnection();

    try {
        connection.setAutoCommit(false);
        PreparedStatement childStatement = connection.prepareStatement(
                "DELETE FROM child WHERE fooId = ?");
        PreparedStatement otherStatement = connection.prepareStatement(
                "DELETE FROM otherfoo WHERE fooId = ?");
        PreparedStatement userStatement = connection.prepareStatement(
                "DELETE FROM userfoo WHERE fooId = ?");
        PreparedStatement modStatement = connection.prepareStatement(
                "DELETE FROM modification WHERE fooId = ?");
        PreparedStatement fooStatement = connection.prepareStatement(
                "DELETE FROM foo WHERE id = ?");
        for (int fooId : fooIds) {
            childStatement.setInt(1, fooId);
            otherStatement.setInt(1, fooId);
            userStatement.setInt(1, fooId);
            modStatement.setInt(1, fooId);
            fooStatement.setInt(1, fooId);
            childStatement.addBatch();
            otherStatement.addBatch();
            userStatement.addBatch();
            modStatement.addBatch();
            fooStatement.addBatch();
        }
        executeBatchAndCheckResult(childStatement, fooIds.length);
        executeBatchAndCheckResult(otherStatement, fooIds.length);
        executeBatchAndCheckResult(userStatement, fooIds.length);
        executeBatchAndCheckResult(modStatement, fooIds.length);
        executeBatchAndCheckResult(fooStatement, fooIds.length);

        connection.commit();
    } catch (SQLException e) {
        connection.rollback();
        throw e;
    } finally {
        connection.close();
    }
}

我也试过这个：

public void deleteFooList2(final int[] fooIds)
    throws SQLException {
    StringBuilder deleteChildSql = new StringBuilder(
            "DELETE FROM child WHERE fooId IN (");
    StringBuilder deleteOtherSql = new StringBuilder(
            "DELETE FROM otherfoo WHERE fooId IN (");
    StringBuilder deleteUserSql = new StringBuilder(
            "DELETE FROM userfoo WHERE fooId IN (");
    StringBuilder deleteModSql = new StringBuilder(
            "DELETE FROM modification WHERE fooId IN (");
    StringBuilder deleteFooSql = new StringBuilder(
            "DELETE FROM foo WHERE id IN (");
    Connection connection = childSource.getConnection();

    try {
        connection.setAutoCommit(false);
        Statement statement = connection.createStatement();
        for (int x = 0; x < fooIds.length; x++) {
            if (x > 0) {
                deleteChildSql.append(",");
                deleteOtherSql.append(",");
                deleteUserSql.append(",");
                deleteModSql.append(",");
                deleteFooSql.append(",");
            }
            deleteChildSql.append(fooIds[x]);
            deleteOtherSql.append(fooIds[x]);
            deleteUserSql.append(fooIds[x]);
            deleteModSql.append(fooIds[x]);
            deleteFooSql.append(fooIds[x]);
        }
        deleteChildSql.append(")");
        deleteOtherSql.append(")");
        deleteUserSql.append(")");
        deleteModSql.append(")");
        deleteFooSql.append(")");
        statement.addBatch(deleteChildSql.toString());
        statement.addBatch(deleteOtherSql.toString());
        statement.addBatch(deleteUserSql.toString());
        statement.addBatch(deleteModSql.toString());
        statement.addBatch(deleteFooSql.toString());
        executeBatchAndCheckResult(statement, fooIds.length);
        connection.commit();
    } catch (SQLException e) {
        connection.rollback();
        throw e;
    } finally {
        connection.close();
    }

使用此辅助函数，包括完整性而非相关性：

private void executeBatchAndCheckResult(Statement statement,
                                        int count)
        throws SQLException {
    int[] results = statement.executeBatch();
    if (results == null) {
        throw new SQLException(
                "Batch update failed to return results!");
    }
    int total = 0;
    for (int result : results) {
        total += result;
        if (result == Statement.EXECUTE_FAILED) {
            String sql = statement.toString();
            throw new SQLException(String.format(
                    "Error executing batch: %s", sql));
        }
    }
    log.info(String.format("Ran batch, statement count %d, row count %d",
            results.length, total));
    if (results.length != count) {
        throw new SQLException(String.format(
                "Batch update failed to execute correct count! " +
                        "(%d instead of %d)", results.length, count));
    }
}

我很惊讶地发现普通Statement的表现比5 PreparedStatements的表现要快得多。事实上，它是不可见的。 Statement与其中PreparedStatements之一一样快。我是否错误地实施了PreparedStatement批次？

这是来自Statement：

的快速SQL

DELETE FROM foo WHERE id IN (52000,52001,52002,52003,52004,52005,52006,52007,52008,52009,52010) 
DELETE FROM modification WHERE fooId IN (52000,52001,52002,52003,52004,52005,52006,52007,52008,52009,52010) 
DELETE FROM userfoo WHERE fooId IN (52000,52001,52002,52003,52004,52005,52006,52007,52008,52009,52010) 
DELETE FROM otherfoo WHERE fooId IN (52000,52001,52002,52003,52004,52005,52006,52007,52008,52009,52010) 
DELETE FROM childfoo WHERE fooId IN (52000,52001,52002,52003,52004,52005,52006,52007,52008,52009,52010)

这是来自PreparedStatements的慢速SQL：

DELETE FROM foo WHERE id = 52010
DELETE FROM modification WHERE fooId = 52010 
DELETE FROM userfoo WHERE fooId = 52010
DELETE FROM otherfoo WHERE fooId = 52010     
DELETE FROM childfoo WHERE fooId = 52010

这是来自P6Spy所以它并不完全是正在发生的事情。来自PreparedStatements的日志记录仅显示添加到批处理的最后一项，而不是所有DELETE。

我确实有MySQL JDBC参数rewriteBatchedStatements=true，所以我假设MySQL Connector / J正在将批次重写为类似的东西，如

DELETE FROM foo WHERE id = 52010 OR id = 52009 OR id = 52008 OR id = 52007 OR id = 52006 OR id = 52005 etc

或者它可能不是？还有什么我可能做错了？

Answer 1

我确实有MySQL JDBC参数rewriteBatchedStatements=true，所以我假设MySQL Connector / J正在重写[DELETE]批次

不，不是。 rewriteBatchedStatements=true仅适用于INSERT INTO ... VALUES ...批次。一批DELETE ... WHERE fooId = ?语句仍将单独发送每个DELETE语句。（通过测试和检查常规日志确认。）这就是为什么您看到PreparedStatement批处理和Statement（已经手动优化以在一次往返中删除多行）之间的性能差异。

PreparedStatement批处理与plain语句的表现不同

1 个答案: