我已经创建:https://jira.spring.io/browse/BATCH-2778
我正在开发 Spring Batch + Redis(Spring数据Redis)示例。在此示例中,我正在读取student.csv
文件并将所有数据存储在 Redis 数据库中。我想将dateOfBirth
用作Date
,并且确定需要进行一些日期逻辑转换以将值Date存储在Redis
中。
根据我的分析,看来我将无法使用@JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd")
,因为我根本不处理JSON。我认为我应该使用Converter等将Byte[]
转换为Date
,将Date
转换为Byte[]
等。
事实上,我尝试了这些选项,但是没有用。有什么帮助吗?
注意:我不想将 dateOfBirth 保存为字符串值。
org.springframework.batch.item.file.FlatFileParseException: Parsing error at line: 2 in resource=[URL [file:c:/Videos/student.csv]], input=[1,John,Doe,05-12-1988 12:34:45]
at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:184) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:89) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader$$FastClassBySpringCGLIB$$ebb633d0.invoke(<generated>) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:746) ~[spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136) ~[spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124) ~[spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) [spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) ~[spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.batch.item.file.FlatFileItemReader$$EnhancerBySpringCGLIB$$87932856.read(<generated>) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:91) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.read(SimpleChunkProvider.java:157) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:116) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:375) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:145) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.provide(SimpleChunkProvider.java:110) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:69) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) ~[spring-tx-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:272) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:375) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:145) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:394) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:308) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:141) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:134) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_162]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_162]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_162]
at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_162]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) [spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) [spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) [spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) [spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at com.sun.proxy.$Proxy69.run(Unknown Source) [na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:163) [spring-boot-autoconfigure-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179) [spring-boot-autoconfigure-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134) [spring-boot-autoconfigure-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128) [spring-boot-autoconfigure-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:792) [spring-boot-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:776) [spring-boot-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1242) [spring-boot-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1230) [spring-boot-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at com.myexample.DateBatchPocApplication.main(DateBatchPocApplication.java:12) [classes/:na]
Caused by: org.springframework.validation.BindException: org.springframework.validation.BeanPropertyBindingResult: 1 errors
Field error in object 'target' on field 'dateOfBirth': rejected value [05-12-1988 12:34:45]; codes [typeMismatch.target.dateOfBirth,typeMismatch.dateOfBirth,typeMismatch.java.util.Date,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [target.dateOfBirth,dateOfBirth]; arguments []; default message [dateOfBirth]]; default message [Failed to convert property value of type 'java.lang.String' to required type 'java.util.Date' for property 'dateOfBirth'; nested exception is java.lang.IllegalStateException: Cannot convert value of type 'java.lang.String' to required type 'java.util.Date' for property 'dateOfBirth': no matching editors or conversion strategy found]
at org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper.mapFieldSet(BeanWrapperFieldSetMapper.java:200) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.item.file.mapping.DefaultLineMapper.mapLine(DefaultLineMapper.java:43) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:180) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
... 56 common frames omitted
StudentJob.java
@Configuration
public class StudentJob {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Autowired
private FlatFileItemReader<Student> studentReader;
@Autowired
private StudentWritter studentWritter;
@Bean
public Job readstudentCSVFileJob() {
return jobBuilderFactory.get("readstudentCSVFileJob").incrementer(new RunIdIncrementer())
.start(countryCurrStepOne()).build();
}
@Bean
public Step countryCurrStepOne() {
return stepBuilderFactory.get("studentStepOne").<Student, Student>chunk(5).reader(studentReader)
.writer(studentWritter).build();
}
}
StudentBatchConfig.java
@Slf4j
@Configuration
public class StudentBatchConfig {
@Bean(destroyMethod="")
@StepScope
public FlatFileItemReader<Student> studentReader(@Value("${input.student.path}") Resource resource) throws IOException {
log.debug("Resource Path : "+resource.getFile());
FlatFileItemReader<Student> itemReader = new FlatFileItemReader<>();
itemReader.setName("STUDENT_READER");
itemReader.setResource(resource);
itemReader.setLineMapper(studentlineMapper());
itemReader.setLinesToSkip(1);
return itemReader;
}
@Bean
public LineMapper<Student> studentlineMapper() {
// Delimited Line Tokenizer
DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer();
lineTokenizer.setNames("id", "firstName", "lastName", "dateOfBirth");
lineTokenizer.setIncludedFields(new int[] {0,1,2,3});
// Bean Wrapper Field SetMapper
BeanWrapperFieldSetMapper<Student> fieldSetMapper = new BeanWrapperFieldSetMapper<>();
fieldSetMapper.setTargetType(Student.class);
DefaultLineMapper<Student> lineMapper = new DefaultLineMapper<>();
lineMapper.setLineTokenizer(lineTokenizer);
lineMapper.setFieldSetMapper(fieldSetMapper);
return lineMapper;
}
@Bean
public StudentWritter studentWritter() {
return new StudentWritter();
}
}
StudentWritter.java
public class StudentWritter implements ItemWriter<Student>{
@Autowired
private StudentRepository studentRepository;
@Override
public void write(List<? extends Student> students) throws Exception {
studentRepository.saveAll(students);
}
}
EDIT-1:
@Slf4j
@Configuration
public class StudentBatchConfig {
private static final SimpleDateFormat format = new SimpleDateFormat("dd-MM-yyyy HH:mm");
@Bean(destroyMethod="")
@StepScope
public FlatFileItemReader<Student> countryReader(@Value("${input.student.path}") Resource resource) throws IOException {
log.debug("Resource Path : "+resource.getFile());
FlatFileItemReader<Student> itemReader = new FlatFileItemReader<>();
itemReader.setName("STUDENT_READER");
itemReader.setResource(resource);
itemReader.setLineMapper(studentLineMapper());
itemReader.setLinesToSkip(1);
return itemReader;
}
@SuppressWarnings("rawtypes")
@Bean
public LineMapper<Student> studentLineMapper() {
DefaultLineMapper<Student> lineMapper = new DefaultLineMapper<>();
// Delimited Line Tokenizer
DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer();
lineTokenizer.setNames("id", "firstName", "lastName", "dateOfBirth");
lineTokenizer.setIncludedFields(new int[] {0,1,2,3});
// Date parsing logic has been added
CustomDateEditor customDateEditor = new CustomDateEditor(format, false);
HashMap<Class, PropertyEditor> customEditors = new HashMap<>();
customEditors.put(Date.class, customDateEditor);
// Bean Wrapper Field SetMapper
BeanWrapperFieldSetMapper<Student> fieldSetMapper = new BeanWrapperFieldSetMapper<>();
fieldSetMapper.setTargetType(Student.class);
fieldSetMapper.setCustomEditors(customEditors);
lineMapper.setLineTokenizer(lineTokenizer);
lineMapper.setFieldSetMapper(fieldSetMapper);
return lineMapper;
}
@Bean
public StudentWritter studentWritter() {
return new StudentWritter();
}
}
Student.java
@Builder
@Data
@AllArgsConstructor
@NoArgsConstructor
@RedisHash("student")
public class Student {
@Id @Indexed
private String id;
private String firstName;
private String lastName;
private Date dateOfBirth;
}
但是日期不能正确存储。.似乎是格式问题。
注意:如果我在Microsoft的CSV编辑器中打开此文件,则会显示dd-MM-yyyy HH:mm
但是如果我在Notepad ++中编辑此文件,则显示为
id,firstName,lastName,dateOfBirth
1,John,Doe,25-11-2018 14:48:10
2,Sameer,Kumbhare,25-11-2018 14:48:10
答案 0 :(得分:0)
您的问题与JSON不相关,根据堆栈跟踪的问题是BeanWrapperFieldSetMapper
不知道如何将05-12-1988 12:34:45
转换为java.util.Date
。您需要使用以下自定义日期编辑器对其进行配置:
CustomDateEditor customDateEditor = new CustomDateEditor(/* your DateFormat here */);
HashMap<Class, PropertyEditor> customEditors = new HashMap<>();
customEditors.put(Date.class, customDateEditor);
fieldSetMapper.setCustomEditors(customEditors);
CustomDateEditor
来自Spring框架:org.springframework.beans.propertyeditors.CustomDateEditor
。
希望这会有所帮助。
答案 1 :(得分:0)
答案在这里发布:https://jira.spring.io/browse/BATCH-2778
或
我使用了类似下面的内容。看起来Redis在Java7 Date API上不能很好地工作,但是在Java 8 API上可以很好地工作。但是没有现成的支持。
public class StudentFieldSetMapper implements FieldSetMapper<Student>{
@Override
public Student mapFieldSet(FieldSet fieldSet) throws BindException {
return Student.builder()
.id(fieldSet.readString("id"))
.firstName(fieldSet.readString("firstName"))
.lastName(fieldSet.readString("lastName"))
.dateOfBirth(fieldSet.readDate("dateOfBirth", "dd-MM-yyyy HH:mm:ss")).build();
}
}