Question

我正在读取csv文件，并使用spring batch（读取，处理和写入）将数据插入数据库。我正在itemWriter类中使用“ jpaRepository.save”将数据保存到数据库中。我试图在@OnSkipInWrite方法中捕获跳过的项目和跳过的消息，但是即使数据被跳过也不会调用此方法。并在batch_step_execution表中： read_count = 18，write_count = 10，write_skip_count = 0，roll_back_count = 8。为什么write_skip_count为0？我只想知道哪个项目被跳过了，什么是例外消息。我的步骤：

@Bean
public Step step() throws IOException {
    return stepBuilderFactory.get("step").<Entity, Entity>chunk(1).reader(multiResourceItemReader())
            .processor(processor()).writer(writer()).faultTolerant().skip(Exception.class).skipLimit(100)
            .listener(new stepExecutionListener()).build();
}

这是我的侦听器类。

     public class StepExecutionListener{

private static final Logger LOG = Logger.getLogger(StepExecutionListener.class);

@OnSkipInRead
public void onSkipInRead(Throwable t) {
    LOG.error("On Skip in Read Error : " + t.getMessage());
}

@OnSkipInWrite
public void onSkipInWrite(Entity item, Throwable t) {
    LOG.error("Skipped in write due to : " + t.getMessage());
}

@OnSkipInProcess
public void onSkipInProcess(Entity item, Throwable t) {
    LOG.error("Skipped in process due to: " + t.getMessage());
}

@OnWriteError
public void onWriteError(Exception exception, List<? extends Entity> items) {
    LOG.error("Error on write on " + items + " : " + exception.getMessage());
}}

为什么不调用@OnSkipInWrite和@OnWriteError？任何帮助将非常感激。预先感谢。

Answer 1

我从您分享的内容中看不到为什么不调用跳过监听器，但这是使用监听器的一个独立示例：

import java.util.Arrays;
import java.util.List;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.StepExecution;
import org.springframework.batch.core.annotation.OnSkipInProcess;
import org.springframework.batch.core.annotation.OnSkipInRead;
import org.springframework.batch.core.annotation.OnSkipInWrite;
import org.springframework.batch.core.annotation.OnWriteError;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.support.ListItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableBatchProcessing
public class MyJob {

    @Autowired
    private JobBuilderFactory jobs;

    @Autowired
    private StepBuilderFactory steps;

    @Bean
    public ItemReader<Integer> itemReader() {
        return new ListItemReader<>(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10));
    }

    @Bean
    public ItemWriter<Integer> itemWriter() {
        return items -> {
            for (Integer item : items) {
                if (item.equals(3)) {
                    throw new Exception("No 3 here!");
                }
                System.out.println("item = " + item);
            }
        };
    }

    @Bean
    public Step step() {
        return steps.get("step")
                .<Integer, Integer>chunk(5)
                .reader(itemReader())
                .writer(itemWriter())
                .faultTolerant()
                .skip(Exception.class)
                .skipLimit(10)
                .listener(new StepExecutionListener())
                .build();
    }

    @Bean
    public Job job() {
        return jobs.get("job")
                .start(step())
                .build();
    }

    public class StepExecutionListener {

        @OnSkipInRead
        public void onSkipInRead(Throwable t) {
            System.err.println("On Skip in Read Error : " + t.getMessage());
        }

        @OnSkipInWrite
        public void onSkipInWrite(Integer item, Throwable t) {
            System.err.println("Skipped in write due to : " + t.getMessage());
        }

        @OnSkipInProcess
        public void onSkipInProcess(Integer item, Throwable t) {
            System.err.println("Skipped in process due to: " + t.getMessage());
        }

        @OnWriteError
        public void onWriteError(Exception exception, List<? extends Integer> items) {
            System.err.println("Error on write on " + items + " : " + exception.getMessage());
        }}

    public static void main(String[] args) throws Exception {
        ApplicationContext context = new AnnotationConfigApplicationContext(MyJob.class);
        JobLauncher jobLauncher = context.getBean(JobLauncher.class);
        Job job = context.getBean(Job.class);
        JobExecution jobExecution = jobLauncher.run(job, new JobParameters());
        StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next();
        System.out.println("WriteSkipCount = " + stepExecution.getWriteSkipCount());
    }

}

此示例打印：

item = 1
item = 2
Error on write on [1, 2, 3, 4, 5] : No 3 here!
item = 1
item = 2
Error on write on [3] : No 3 here!
item = 4
Skipped in write due to : No 3 here!
item = 5
item = 6
item = 7
item = 8
item = 9
item = 10
WriteSkipCount = 1

这意味着在写入时跳过某项并且writeSkipCount是正确的时，将调用跳过侦听器。

希望这会有所帮助。

Answer 2

您可以实现SkipListener接口，而不使用@OnWriteError注释。

Answer 3

在您的 BatchConf 中尝试：

@Bean
@StepScope
public StepExecutionListener stepExecutionListener() {
    return new StepExecutionListener();
}

... .skipLimit(1) .listener(stepExecutionListener()
.build();

在SkipListener中未调用@OnSkipInWrite

3 个答案: