我们正在使用Spring Batch将管道分隔文件ETL到DB中。文件中的每条记录都有许多字段，并由ClaimNumber标识：
ClaimNumber|AdjustmentVersion|.....
0038017282|3|....
0071517729|3|....
0081517745|3|....
在批处理步骤中，使用常规的读取 - 处理 - 写入流程：
<step id="stagingDataDump" next="gatherStats">
<tasklet>
<!-- <chunk reader="genericBatchItemReader" writer="genericBatchItemWriter" -->
<chunk reader="genericBatchItemReader" writer="compositeWriter" processor="validationProcessor"
commit-interval="1000" skip-limit="100000" >
<skippable-exception-classes>
<batch:include class="org.springframework.batch.item.file.FlatFileParseException" />
<batch:include class="org.beanio.BeanIOException" />
</skippable-exception-classes>
</chunk>
<listeners>
<listener ref="genericItemSkipListener"/>
</listeners>
</tasklet>
</step>
Reader使用BeanIOFlatFileItemReader：
<bean id="genericBatchItemReader" class="org.beanio.spring.BeanIOFlatFileItemReader" scope="step"
p:streamMapping="classpath:beanio-mapping.xml"
p:streamName="#{jobParameters[feedProcessorLauncherImpl.BEANIO_STREAM_MAPPING]}"
p:resource="file://#{jobParameters[feedProcessorLauncherImpl.RESOURCE_FILE_NAME_UNENCRYPTED]}"
p:errorHandler-ref="beanIoRecordErrorHandler"/>
处理器阶段封装了项目验证：
<util:map id="handlerRegistryContents">
<entry key="#{T(org.fuwt.iws.claims.service.filemanagement.ClaimsEnums$ContentSubType).MEDI}" value-ref="medicalClaimsValidator"/>
<entry key="#{T(org.fuwt.iws.claims.service.filemanagement.ClaimsEnums$ContentSubType).LAB}" value-ref="labClaimsValidator"/>
<entry key="#{T(org.fuwt.iws.claims.service.filemanagement.ClaimsEnums$ContentSubType).RXPD}" value-ref="pharmaClaimsValidator"/>
</util:map>
<bean id="validationProcessor" class="org.fuwt.iws.claims.validation.springbatch.ValidationProcessor" scope="step">
<property name="handlerRegistry" ref="handlerRegistryContents"/>
</bean>
写是复合的：
<bean id="genericBatchItemWriter" class="org.fuwt.iws.claims.springbatch.GenericBatchItemWriter" scope="step"
p:metadataId="#{jobParameters[feedProcessorLauncherImpl.METADATA_ID]}"/>
<bean id="softValidationsItemWriter" class="org.fuwt.iws.claims.springbatch.SoftValidationsItemWriter" scope="step"
p:metadataId="#{jobParameters[feedProcessorLauncherImpl.METADATA_ID]}"/>
<bean id="compositeWriter" class="org.springframework.batch.item.support.CompositeItemWriter" scope="step">
<property name="delegates">
<list>
<!-- Order here is significant as ID's, which are generated by the first writer - genericBatchItemWriter - need to be passed around -->
<ref bean="genericBatchItemWriter"/>
<ref bean="softValidationsItemWriter"/>
</list>
</property>
</bean>
在上面的处理/验证步骤中，ValidationProcessor确定记录的类型，根据该类型实例化适当的复合验证器（在这种情况下为MedicalClaimsValidator），其中所有个体在下面的日志中配置了此类型的验证（复合模式），HCPCSCodeLength。
虽然通过此基础架构验证了每个项目（声明），但发现的错误会累积到项目errors字段中 - Map<String, Collection<String>>，其中特定验证的失败由此键入验证名称并在地图的值中进行了描述。
这让我想到了我们所看到的错误行为：
在仔细检查上述测试CSV文件的负载后，对3个索赔记录进行了日志文件检查，我们发现了以下（奇怪的！？）行为，即每条记录的验证次数与文件中的序号一样多。如下所示：第一个记录（声明）被验证一次，导致错误映射中的单个记录;第二个记录（声明）被验证两次，它的错误映射现在包含重复两次的消息;第3条记录正在被验证3次，导致进入错误地图，同样的信息重复3次。
文件中的每条记录在无效性方面几乎完全相同，因此预期的结果是每条记录应该具有相同的错误集合。
实际结果是错误的值随着每个后续记录而不断增加：
INFO 2016-06-23 10:16:24,214 [main] org.fuwt.iws.claims.validation.springbatch.medical.MedicalClaimsValidator: Service date to: Thu Dec 10 00:00:00 EST 2015
INFO 2016-06-23 10:16:24,216 [main] org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength: Started HCPCSCodeLength validation on org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength@1a9ddcb7 for claimNumber = 0038017282
INFO 2016-06-23 10:16:24,223 [main] org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength: Errors: {HCPCSCode=[supplied HCPCS code is blank or null, but should be 3-5 characters in length, anything greater than or less than is invalid value]}
INFO 2016-06-23 10:16:24,227 [main] org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength: Started HCPCSCodeLength validation on org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength@1a9ddcb7 for claimNumber = 0071517729
INFO 2016-06-23 10:16:24,228 [main] org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength: Errors: {HCPCSCode=[supplied HCPCS code is blank or null, but should be 3-5 characters in length, anything greater than or less than is invalid value]}
INFO 2016-06-23 10:16:24,228 [main] org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength: Started HCPCSCodeLength validation on org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength@1a9ddcb7 for claimNumber = 0071517729
INFO 2016-06-23 10:16:24,228 [main] org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength: Errors: {HCPCSCode=[supplied HCPCS code is blank or null, but should be 3-5 characters in length, anything greater than or less than is invalid value, supplied HCPCS code is blank or null, but should be 3-5 characters in length, anything greater than or less than is invalid value]}
INFO 2016-06-23 10:16:24,228 [main] org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength: Started HCPCSCodeLength validation on org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength@1a9ddcb7 for claimNumber = 0081517745
INFO 2016-06-23 10:16:24,228 [main] org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength: Errors: {HCPCSCode=[supplied HCPCS code is blank or null, but should be 3-5 characters in length, anything greater than or less than is invalid value]}
INFO 2016-06-23 10:16:24,228 [main] org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength: Started HCPCSCodeLength validation on org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength@1a9ddcb7 for claimNumber = 0081517745
INFO 2016-06-23 10:16:24,228 [main] org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength: Errors: {HCPCSCode=[supplied HCPCS code is blank or null, but should be 3-5 characters in length, anything greater than or less than is invalid value, supplied HCPCS code is blank or null, but should be 3-5 characters in length, anything greater than or less than is invalid value]}
INFO 2016-06-23 10:16:24,228 [main] org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength: Started HCPCSCodeLength validation on org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength@1a9ddcb7 for claimNumber = 0081517745
INFO 2016-06-23 10:16:24,229 [main] org.fuwt.iws.claims.validation.springbatch.medical.HCPCSCodeLength: Errors: {HCPCSCode=[supplied HCPCS code is blank or null, but should be 3-5 characters in length, anything greater than or less than is invalid value, supplied HCPCS code is blank or null, but should be 3-5 characters in length, anything greater than or less than is invalid value, supplied HCPCS code is blank or null, but should be 3-5 characters in length, anything greater than or less than is invalid value]}
使用的版本：
spring-batch-core: 2.2.0.RELEASE
beanio: 2.1.0
如果有什么东西让Spring Batch在处理器上发出那些重复的调用，那么Spring Batch的正常行为是什么，以及如何阻止它并实现我上面描述的所需行为呢？
此验证组件表现出错误的行为：
@Component("medicalClaimsValidator")
public class MedicalClaimsValidator implements ClaimValidation {
private final static Logger logger = LoggerFactory.getLogger(MedicalClaimsValidator.class);
@Autowired private AbstractMedicalClaimValidation HCPCSCodeLength;
List<ClaimValidation> medicalClaimValidations = new ArrayList<>();
@Override
public boolean supports(Class<?> clazz) {
return QualcareMedicalClaimWeeklyNDT.class.equals(clazz);
}
@Override
public Map<String, Collection<String>> validate(Object item, MessageSource messageSource) {
logger.info("\nSoft-validating the bean...");
QualcareMedicalClaimWeeklyNDT medicalClaim = (QualcareMedicalClaimWeeklyNDT)item;
logger.info("Claim #: {}", medicalClaim.getClaimNumber());
logger.info("Service date from: {}", medicalClaim.getServiceDateFrom());
logger.info("Service date to: {}", medicalClaim.getServiceDateTo());
//TODO: A candidate for externalization into a config file once we have all the known rules
//medicalClaimValidations.add(new ServiceDateFromGreaterThanTo());
//medicalClaimValidations.add(new ProcedureCodeLength());
medicalClaimValidations.add(HCPCSCodeLength/*new HCPCSCodeLength()*/);
//medicalClaimValidations.add(new TypeOfBillPresenseAndLengthForInstitutionalClaims());
//medicalClaimValidations.add(new DischargeStatusPresenseAndLengthForInpatientClaims());
//medicalClaimValidations.add(new DiagnosisCodeFormat());
for(ClaimValidation validation:medicalClaimValidations) {
logger.info("validation type: {}",validation.getClass());
validation.validate(medicalClaim, messageSource);
}
return medicalClaim.getErrors();
}
}
以下解决方法隐藏了错误行为：
@Component("medicalClaimsValidator")
public class MedicalClaimsValidator implements ClaimValidation {
private final static Logger logger = LoggerFactory.getLogger(MedicalClaimsValidator.class);
@Autowired @Qualifier("HCPCSCodeLength")private AbstractMedicalClaimValidation HCPCSCodeLength;
@Autowired @Qualifier("serviceDateFromGreaterThanTo")private AbstractMedicalClaimValidation serviceDateFromGreaterThanTo;
@Autowired @Qualifier("procedureCodeLength")private AbstractMedicalClaimValidation procedureCodeLength;
@Autowired @Qualifier("typeOfBillPresenseAndLengthForInstitutionalClaims")private AbstractMedicalClaimValidation typeOfBillPresenseAndLengthForInstitutionalClaims;
@Autowired @Qualifier("dischargeStatusPresenseAndLengthForInpatientClaims")private AbstractMedicalClaimValidation dischargeStatusPresenseAndLengthForInpatientClaims;
@Autowired @Qualifier("diagnosisCodeFormat")private AbstractMedicalClaimValidation diagnosisCodeFormat;
List<ValidationProcessTuple> medicalClaimValidations = new ArrayList<>();
@Override
public boolean supports(Class<?> clazz) {
return QualcareMedicalClaimWeeklyNDT.class.equals(clazz);
}
@Override
public Map<String, Collection<String>> validate(Object item, MessageSource messageSource) {
logger.info("\nSoft-validating the bean...");
QualcareMedicalClaimWeeklyNDT medicalClaim = (QualcareMedicalClaimWeeklyNDT)item;
logger.info("Claim #: {}", medicalClaim.getClaimNumber());
logger.info("Service date from: {}", medicalClaim.getServiceDateFrom());
logger.info("Service date to: {}", medicalClaim.getServiceDateTo());
//TODO: A candidate for externalization into a config file once we have all the known rules
medicalClaimValidations.add(new ValidationProcessTuple(serviceDateFromGreaterThanTo, false));
medicalClaimValidations.add(new ValidationProcessTuple(procedureCodeLength, false));
medicalClaimValidations.add(new ValidationProcessTuple(HCPCSCodeLength, false));
medicalClaimValidations.add(new ValidationProcessTuple(typeOfBillPresenseAndLengthForInstitutionalClaims, false));
medicalClaimValidations.add(new ValidationProcessTuple(dischargeStatusPresenseAndLengthForInpatientClaims, false));
medicalClaimValidations.add(new ValidationProcessTuple(diagnosisCodeFormat, false));
for (ValidationProcessTuple tuple : medicalClaimValidations) {
if (!tuple.processed) {//to counteract the erroneous behavior whereby validation calls get repeated as many times as there are records
tuple.validation.validate(item, messageSource);
tuple.processed = true;
}
}
return medicalClaim.getErrors();
}
}
我仍然处于黑暗中，为什么这种行为首先发生 - 对此的任何解释当然都是受欢迎的。
答案 0 :(得分：2)
在MedicalClaimsValidator类中，为什么要在validate（）方法中累积此List中的读取项？我没有看到原因。每次处理新行时，它都会在此列表中不断添加已处理的元组。您可以在方法，init方法或构造函数中定义处理规则。
//TODO: A candidate for externalization into a config file once we have all the known rules
medicalClaimValidations.add(new ValidationProcessTuple(serviceDateFromGreaterThanTo, false));
medicalClaimValidations.add(new ValidationProcessTuple(procedureCodeLength, false));
medicalClaimValidations.add(new ValidationProcessTuple(HCPCSCodeLength, false));
medicalClaimValidations.add(new ValidationProcessTuple(typeOfBillPresenseAndLengthForInstitutionalClaims, false));
medicalClaimValidations.add(new ValidationProcessTuple(dischargeStatusPresenseAndLengthForInpatientClaims, false));
medicalClaimValidations.add(new ValidationProcessTuple(diagnosisCodeFormat, false));