我有一个 spring 批处理作业,我必须检查所有文件行中的 id 是否相等,并且应该跳过包含不同 id 的行。我所做的是保存第一条记录,然后比较每行的 id,如果 id 不同,则抛出运行时异常,但由于某种原因,spring 批处理工作,直到它得到“要排除”的行,然后重复写入通过在异常上写入所有记录来处理。 这就是我的意思:
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
2021-06-03 11:12:28.466 ERROR 41416 --- [ scheduling-1] tn.itserv.batch.SkipLinesListener : An error occured while writing the input Force rollback on skippable exception so that skipped item can be located.
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=7798661]
我的步骤:
@Bean
public Step loadFiles() throws IOException {
return stepBuilderFactory
.get("step1")
.<FileModelIn, FileModelOut>chunk(100)
.reader(multiResourceItemReader())
.processor(batchProcessor())
.writer(batchWriter())
.faultTolerant()
.skipPolicy(skipLinesListener())
.noRetry(RuntimeException.class)
.noRollback(RuntimeException.class)
.listener(MyStepListner())
.build();
}
跳过策略:
public class SkipLinesListener implements SkipPolicy {
private static final int MAX_SKIP_COUNT = 10;
private static final Logger logger = LoggerFactory.getLogger(SkipLinesListener.class);
@Override
public boolean shouldSkip(Throwable t, int skipCount) throws SkipLimitExceededException {
if (t instanceof RuntimeException && skipCount < MAX_SKIP_COUNT )
{ RuntimeException ex=(RuntimeException)t;
logger.error("An error occured while writing the input "+ ex.getMessage());
return true;
}
if (t instanceof FlatFileParseException && skipCount < MAX_SKIP_COUNT ) {
FlatFileParseException ex = (FlatFileParseException) t;
logger.error("An error occured while processing the "+ ex.getInput());
return true;
}
return false;
}
}
我不知道为什么会出现这种行为,我是不是遗漏了什么? 我在 itemwriter 类中抛出异常
@Override
public void write(List<? extends FileModelOut> items) throws Exception {
List<FCCampaignModel> campaigns=new ArrayList<FCCampaignModel>();
List<sms> smsList=new ArrayList<>();
FCCampaignModel firstLine=cmsDaoProxy.addCampaign(items.get(0).getFcCampaignModel());
for (FileModelOut fileContent : items) {
if (fileContent.getFcCampaignModel().getId_Campaign().equals(firstLine.getId_Campaign()))
{
smsRepository.save(fileContent.getSms());
}
else throw new RuntimeException("different id campaign detected : "+fileContent.getFcCampaignModel().getId_Campaign());
}
答案 0 :(得分:2)
您的异常被声明为可跳过的异常,因此当它从项目编写器抛出时,Spring Batch 将逐项扫描块,即一项一项地重新处理项,每个项都在其自己的事务中。
>这是因为items是分块写入的(即bulk模式),如果在那个bulk-write操作中出现异常,Spring Batch无法知道是哪个item导致了问题,所以会一一重试。您可以在示例模块中找到一个示例:Chunk Scanning Sample。