spring batch2.1.9 - 分隔符bug

时间:2017-05-08 18:51:17

标签: csv spring-batch delimiter

我使用的是spring batch 2.1.9(classpath中有两个jar-spring-core-core-2.2.1.RELEASE.jar和spring-batch-infrastructure-2.1.9.RELEASE.jar)

我有一个csv文件作为输入,我想将数据存储在数据库中,并获得4个文件作为输出。

输入文件中的分隔符是;

正确启动第一个插入作业,并将数据插入数据库。

这没关系。第二个作业我采用相同的文件,其数据插入到具有相同分隔符的数据库中。

但是当我运行我的流程时,我立即收到错误异常

INFOS: Executing step: [stepMultiWriters]
mai 08, 2017 8:28:55 PM org.springframework.batch.core.step.AbstractStep execute
GRAVE: Encountered an error executing the step
org.springframework.batch.item.file.FlatFileParseException: Parsing error at line: 1 in resource=[URL [file:C:/Temp/input/input1.csv]], input=[CNRA;COS;CHARAFI;Traitement base;10000;2000]
at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:182)
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:85)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy62.read(Unknown Source)
at org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:90)
at org.springframework.batch.core.step.item.SimpleChunkProvider.read(SimpleChunkProvider.java:150)
at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:110)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:367)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:214)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:143)
at org.springframework.batch.core.step.item.SimpleChunkProvider.provide(SimpleChunkProvider.java:105)
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:68)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:386)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:264)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:76)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:367)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:214)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:143)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:250)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:195)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:135)
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:61)
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:144)
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:124)
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:135)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:293)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:120)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:114)
at ma.controle.gestion.bean.batch.BatchExcelBean.uploadFile(BatchExcelBean.java:78)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.el.parser.AstValue.invoke(AstValue.java:279)
at org.apache.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:273)
at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:105)
at org.primefaces.component.fileupload.FileUpload.broadcast(FileUpload.java:319)
at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:755)
at javax.faces.component.UIViewRoot.processDecodes(UIViewRoot.java:931)
at com.sun.faces.lifecycle.ApplyRequestValuesPhase.execute(ApplyRequestValuesPhase.java:78)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:198)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:646)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.primefaces.barcelona.filter.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:32)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:198)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:958)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:318)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Unknown Source)
Caused by: org.springframework.batch.item.file.transform.IncorrectTokenCountException: Incorrect number of tokens found in record: expected 6 actual 1
at org.springframework.batch.item.file.transform.AbstractLineTokenizer.tokenize(AbstractLineTokenizer.java:123)
at org.springframework.batch.item.file.mapping.DefaultLineMapper.mapLine(DefaultLineMapper.java:42)
at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:179)
... 82 more

mai 08, 2017 8:28:55 PM 
org.springframework.batch.core.launch.support.SimpleJobLauncher$1 run
INFOS: Job: [FlowJob: [name=exampleMultiWritersJob]] completed with the following parameters: [{now=1494271735553, file=C:/Temp/input/input1.csv}] and the following status: [FAILED]
Job1 Exit Status : FAILED

这是我的spring-batch-context

    <!-- JobRepository and JobLauncher are configuration/setup classes -->
    <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean" />

   <bean id="jobLauncher"   class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
    <property name="jobRepository" ref="jobRepository" />
   </bean>

   <!-- à voir comment récuperer le nom du fichier et le mettre au value -->
   <bean id="multiResourceItemReader" class="org.springframework.batch.item.file.MultiResourceItemReader">
    <property name="resources" value="file:C:/Temp/input/input.csv" />
    <property name="delegate" ref="flatFileItemReader" />
   </bean>

  <!-- ItemReader reads a complete line one by one from input file -->
  <bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"  scope="step">

    <property name="lineMapper">

        <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">

            <property name="fieldSetMapper">
                <!-- Mapper which maps each individual items in a record to properties in POJO -->
                <bean class="ma.controle.gestion.springbatch.BatchFieldSetMapper" />
            </property>

            <property name="lineTokenizer">
                <!-- A tokenizer class to be used when items in input record are separated by specific characters -->
                <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                    <property name="delimiter" value=";" />
                </bean>
            </property>

        </bean>

    </property>

  </bean>


   <!-- ItemWriter which writes data to database -->
   <bean id="databaseItemWriter" class="org.springframework.batch.item.database.HibernateItemWriter">
    <property name="sessionFactory" ref="sessionFactory" />
   </bean>





 <bean id="exampleFileSourceReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
  <property name="resource" value="file:#{jobParameters['file']}" />
  <property name="lineMapper">
     <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
        <!-- split it -->
        <property name="lineTokenizer">
            <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                <property name="names" value="institution,type,nom,rubrique,montantPaye,MontantRetenu" />
            </bean>
        </property>
        <property name="fieldSetMapper">
            <!-- map to an object -->
            <bean class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
                <property name="prototypeBeanName" value="exampleFileMapper" />
            </bean>
         </property>
    </bean>
</property>


                                                                                                                                                                                                                                                                                                                        

<property name="lineAggregator">
    <bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
        <property name="delimiter" value=";" />
        <property name="fieldExtractor">
            <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                <property name="names" value="institution,type,nom,rubrique,montantPaye,MontantRetenu" />
            </bean>
        </property>
    </bean>
</property>

 <bean id="rcarCopWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
<!-- write to this csv file -->
<property name="resource" value="file:C:/Temp/output/rcarCop.csv" />
<property name="shouldDeleteIfExists" value="true" />
<property name="shouldDeleteIfEmpty" value="true" />
<property name="appendAllowed" value="true" />

<property name="lineAggregator">
    <bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
        <property name="delimiter" value=";" />
        <property name="fieldExtractor">
            <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                <property name="names" value="institution,type,nom,rubrique,montantPaye,MontantRetenu" />
            </bean>
        </property>
    </bean>
</property>

 <bean id="rcarCosWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
<!-- write to this csv file -->
<property name="resource" value="file:C:/Temp/output/rcarCos.csv" />
<property name="shouldDeleteIfExists" value="true" />
<property name="shouldDeleteIfEmpty" value="true" />
<property name="appendAllowed" value="true" />

<property name="lineAggregator">
    <bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
        <property name="delimiter" value=";" />
        <property name="fieldExtractor">
            <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                <property name="names" value="institution,type,nom,rubrique,montantPaye,MontantRetenu" />
            </bean>
        </property>
    </bean>
</property>

  <!-- Optional ItemProcessor to perform business logic/filtering on the input records -->
<bean id="itemProcessor" class="ma.controle.gestion.springbatch.BatchItemProcessor" />

<!-- Optional JobExecutionListener to perform business logic before and after the job -->
<bean id="jobListener" class="ma.controle.gestion.springbatch.BatchJobItemListener" />

<batch:job id="exampleMultiWritersJob">
    <batch:step id="stepMultiWriters">
        <batch:tasklet transaction-manager="txManager">
            <batch:chunk reader="exampleFileSourceReader" writer="exampleMultiWriters" commit-interval="10">
                <batch:streams>
                    <batch:stream ref="cnraCosWriter" />
                    <batch:stream ref="cnraCopWriter" />
                    <batch:stream ref="rcarCosWriter" />
                    <batch:stream ref="rcarCopWriter" />
                </batch:streams>
            </batch:chunk>
        </batch:tasklet>
    </batch:step>

</batch:job>

<!-- Actual Job -->
<batch:job id="batchJob">
    <batch:step id="step1">
        <batch:tasklet transaction-manager="txManager">
            <batch:chunk reader="multiResourceItemReader" writer="databaseItemWriter"
                processor="itemProcessor" commit-interval="10" />
        </batch:tasklet>
    </batch:step>
    <batch:listeners>
        <batch:listener ref="jobListener" />
    </batch:listeners>
</batch:job>

我不确定我哪里出错了。

任何帮助将不胜感激

感谢。

1 个答案:

答案 0 :(得分:1)

据我所知,在为“exampleFileSourceReader”定义行标记器时,不设置分隔符。默认情况下,它是逗号,但您的文件包含“;”。

“flatFileItemReader”的定义是正确的,并包含分隔符。因此,这个正在发挥作用。

<bean id="exampleFileSourceReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
  <property name="resource" value="file:#{jobParameters['file']}" />
  <property name="lineMapper">
     <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
        <!-- split it -->
        <property name="lineTokenizer">
            <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">

                <!-- this is missing --> 
                <property name="delimiter" value";"/>


                <property name="names" value="institution,type,nom,rubrique,montantPaye,MontantRetenu" />
            </bean>
        </property>
        <property name="fieldSetMapper">
            <!-- map to an object -->
            <bean class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
                <property name="prototypeBeanName" value="exampleFileMapper" />
            </bean>
         </property>
    </bean>
</property>