Question

在JPA中，有没有办法在DB中批量插入数据，只有在DB中不存在时才插入。当我尝试批量插入时，由于唯一键约束，它会抛出异常。我想插入DB中不存在的所有记录，其余部分应该被跳过。

org.springframework.dao.DataIntegrityViolationException：无法执行声明; SQL [不适用];约束 [reference_number_master_id_key];嵌套异常是 org.hibernate.exception.ConstraintViolationException：不能执行声明

Answer 1

我和我的团队最终创建了JpaRepositories扩展的实现，以添加此类行为。

主要界面

public interface BatchInsertable <T> {
    enum Mode {NORMAL, IGNORE, REPLACE}

    void batchInsertUsingMode(Collection<T> items, final Mode mode);
}

这就是我们将它连接到现有JpaRepository的方式。

public interface UrlRepository extends JpaRepository<UrlEntity, Long>, BatchInsertable<UrlEntity> {
    // insert common methods
}

......以及它的用法

urlRepository.batchInsertUsingMode(newUrlEntities, BatchInsertable.Mode.IGNORE);

批量插入器实现

@Component
@Scope("prototype")
@Lazy
public class BatchInserter<T> extends AbstractBaseBatchJdbcMutation<T> implements BatchInsertable<T> {

    @Override
    public void batchInsertUsingMode(final Collection<T> items, final Mode mode) {
        final Map<String, String> parameters = new HashMap<>();
        parameters.put("MODE", mode == Mode.IGNORE ? "INSERT IGNORE" : mode == Mode.REPLACE ? "REPLACE" : "INSERT");
        parameters.put("STAGING_TABLE", getTableName());
        parameters.put("COLUMNS", buildColumnNameList());
        parameters.put("PLACEHOLDERS", buildBindVariablePlaceholderList());

        final StrSubstitutor strSubstitutor = new StrSubstitutor(parameters);
        final String sqlTemplate = "${MODE} INTO `${STAGING_TABLE}` (${COLUMNS}) VALUES (${PLACEHOLDERS})";
        getJdbcTemplate().batchUpdate(strSubstitutor.replace(sqlTemplate), buildBindVariables(items));
    }
}

批量删除器实现

@Component
@Scope("prototype")
@Lazy
public class BatchDeleter<T> extends AbstractBaseBatchJdbcMutation<T> implements BatchDeletable<T> {

    @Override
    public int batchDelete(final List<T> items) {
        final Map<String, String> parameters = new HashMap<>();
        parameters.put("STAGING_TABLE", getTableName());
        parameters.put("COLUMNS", buildColumnNameList());
        parameters.put("PLACEHOLDERS", buildBindVariablePlaceholderList());

        final StrSubstitutor strSubstitutor = new StrSubstitutor(parameters);
        final String sqlTemplate = "DELETE FROM `${STAGING_TABLE}` WHERE (${COLUMNS}) = (${PLACEHOLDERS})";
        final int[] updateCounts = getJdbcTemplate().batchUpdate(strSubstitutor.replace(sqlTemplate), buildBindVariables(items));
        return sum(updateCounts);
    }

    private static int sum(final int[] updateCounts) {
        int sum = 0;

        for (final int updateCount : updateCounts) {
            sum += updateCount;
        }

        return sum;
    }
}

一个抽象类，用于维护常见的配置绑定逻辑

public abstract class AbstractBaseBatchJdbcMutation<T> {
    private JdbcTemplate jdbcTemplate;

    private List<ColumnValueExtractors> columnExtractors;

    private String tableName;

    public void setColumnExtractors(final List<ColumnValueExtractors> columnExtractors) {
        this.columnExtractors = new ArrayList<>(columnExtractors);
    }

    public void setTableName(final String tableName) {
        this.tableName = tableName;
    }

    protected List<Object[]> buildBindVariables(final Collection<T> items) {
        return FluentIterable.from(items).transform(new BulkBindingTransform<T>(columnExtractors)).toList();
    }

    protected JdbcTemplate getJdbcTemplate() {
        return jdbcTemplate;
    }

    protected String getTableName() {
        return tableName;
    }

    @Autowired
    public void setDataSource(final DataSource datasource) {
        this.jdbcTemplate = new JdbcTemplate(datasource);
    }

    protected String buildColumnNameList() {
        return join(extract(columnExtractors, on(ColumnValueExtractors.class).getColumnName()), ",");
    }

    protected String buildBindVariablePlaceholderList() {
        return join(nCopies(columnExtractors.size(), "?"), ",");
    }

    private static class BulkBindingTransform<T> implements Function<T, Object[]> {

        private final List<ColumnValueExtractors> columns;
        private BulkBindingTransform(final List<ColumnValueExtractors> columns) {
            this.columns = columns;
        }

        @Nullable
        @Override
        public Object[] apply(final T input) {
            final Object[] bindings = new Object[columns.size()];

            for (int i = 0; i < columns.size(); i++) {
                bindings[i] = columns.get(i).resolveValue(input);
            }

            return bindings;
        }

    }
}

这也可以让您绕过直接与默认save(Iterable<S> iterable)界面接口可能遇到的一些缓慢。我们将它全部用于批处理SQL操作。令人惊讶的是，这么简单的任务是多么复杂。我敢打赌，你可以减少它以满足你的特定需求。：）

在JPA中，有没有办法在数据库中批量插入数据，只有在数据库

1 个答案: