Question

我有一个springbatch应用程序，使用JdbcPagingItemReader提取所有行时遇到问题。

总行- 从web_view选择count（*）; -- 12057

datasource.chunk.size = 500

 @Bean
        public PagingQueryProvider pagingQueryProvider() {
            SqlServerPagingQueryProvider sqlServerPagingQueryProvider = new SqlServerPagingQueryProvider();
            Map<String, Order> sortKey = new HashMap<>();
            sortKey.put("type", Order.ASCENDING);
            sqlServerPagingQueryProvider.setSelectClause("select *");
            sqlServerPagingQueryProvider.setFromClause("from web_view");
            sqlServerPagingQueryProvider.setSortKeys(sortKey);
            return sqlServerPagingQueryProvider;
        }


 @Bean
    public JdbcPagingItemReader<Web> jdbcPagingItemReader() throws Exception {
        return new JdbcPagingItemReaderBuilder<Web>()
                .name("jdbcPagingItemReader")
                .dataSource(dataSource())
                .queryProvider(pagingQueryProvider())
                .rowMapper(new BeanPropertyRowMapper<>(Web.class))
                .pageSize(chunkSize)
                .saveState(true)
                .build();
    }

@Bean
    public Job importUserJob(JobCompletionNotificationListener jobCompletionNotificationListener, Step step1) {
        return jobBuilderFactory.get("migrateWebJob")
                .incrementer(new RunIdIncrementer())
                .listener(jobCompletionNotificationListener)
                .flow(step1)
                .end()
                .build();
    }
@Bean
    public Step step1() throws Exception {
        return stepBuilderFactory.get("step1")
                .<Web, WebStore>chunk(chunkSize)
                .reader(jdbcPagingItemReader())
                .processor(webStoreItemProcessor())
                .writer(eventHubItemWriter())
                .build();
    }

在第三次分页处理之后，在12057条中只有1500行后，它正常停止。

Answer 1

以下是我找到的解决方法。我使用的“类型”排序键具有3个不同的记录。我不确定这是否会有问题。我在视图中添加了一个额外的标识列，并在排序键中添加了该行以解决问题。

Exp：

create or alter view my_view as
select *, row_number() over (order by (select 1)) row_num
from (select * from tab1
union
select * from tab2
union
select * from tab3)x;


@Bean
    public PagingQueryProvider pagingQueryProvider() throws Exception {
        SqlPagingQueryProviderFactoryBean sqlPagingQueryProviderFactoryBean = new SqlPagingQueryProviderFactoryBean();
        Map<String, Order> sortKey = new HashMap<>();
        sortKey.put("row_num", Order.ASCENDING);
        sqlPagingQueryProviderFactoryBean.setDataSource(dataSource());
        sqlPagingQueryProviderFactoryBean.setSelectClause("select *");
        sqlPagingQueryProviderFactoryBean.setFromClause("from dse.zyntrack_web_view");
        sqlPagingQueryProviderFactoryBean.setSortKeys(sortKey);
        return sqlPagingQueryProviderFactoryBean.getObject();
    }

Answer 2

获取字段密钥类型的工作。

OraclePagingQueryProvider queryProvider = new OraclePagingQueryProvider();
queryProvider.setSelectClause("*");
queryProvider.setFromClause("from MEMENTO_REQUEST");

Map<String, Order> sortKeys = new HashMap<>(1);

sortKeys.put("PROG_REQUEST_ID", Order.ASCENDING);

如果您对不是PK的字段进行排序，则获取的数据将小于表中的所有记录

SpringBatch（JdbcPagingItemReader）不会获取所有行

2 个答案: