无法使用弹簧批处理仅使用逗号分隔的CSV

时间：2015-12-04 19:22:56

标签： java spring csv spring-batch

尝试从FlatFileItemReader解析CSV文件时收到IncorrectTokenCountException .CSV文件仅用逗号分隔，甚至换行符都是逗号。我想提取13个字段，分别是SYMBOL，SERIES，OPEN，HIGH，LOW，CLOSE，LAST ，PREVCLOSE，TOTTRDQTY，TOTTRDVAL，时间戳，TOTALTRADES。

但它正在从输入中读取额外的'，'。 输入=

  

[20微米，EQ，29.75,29.75,28.5,28.75,28.85,29,48937,1421432.2,04-DEC-2015,238，INE144J01027，]

已经回答的问题似乎没有帮助我。我如何确保只有前13个代币被视为单行。

例外：

org.springframework.batch.item.file.FlatFileParseException: Parsing error at line: 2 in resource=[class path resource [cm04DEC2015bhav.csv]], input=[20MICRONS,EQ,29.75,29.75,28.5,28.75,28.85,29,48937,1421432.2,04-DEC-2015,238,INE144J01027,]
at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:183)
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy1.read(Unknown Source)
at org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:91)
at org.springframework.batch.core.step.item.SimpleChunkProvider.read(SimpleChunkProvider.java:155)
at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:114)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:368)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at org.springframework.batch.core.step.item.SimpleChunkProvider.provide(SimpleChunkProvider.java:108)
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:69)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:395)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:267)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:77)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:368)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:253)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:195)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:137)
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64)
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:152)
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:131)
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:135)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:301)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:134)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:49)
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:127)
at com.equities.batch.App.main(App.java:37)
Caused by:                   org.springframework.batch.item.file.transform.IncorrectTokenCountException: Incorrect number of tokens found in record: expected 13 actual 14
at org.springframework.batch.item.file.transform.AbstractLineTokenizer.tokenize(AbstractLineTokenizer.java:124)
at    org.springframework.batch.item.file.mapping.DefaultLineMapper.mapLine(DefaultLineMapper.java:43)
at          org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader     .java:180)
 ... 41 more

示例CSV文件内容： SYMBOL，系列，开盘价，最高价，最低价，收盘价，最后，PREVCLOSE，TOTTRDQTY，TOTTRDVAL，TIMESTAMP，TOTALTRADES，ISIN，20微米，EQ，29.75,29.75,28.5,28.75,28.85,29,48937,1421432.2,04-DEC- 2015,238，INE144J01027,3IINFOTECH，EQ，4.3,5.15,4,5.15,5.15,4.3,16790313,81105043.75,04-DEC-2015,5998，INE748C01020,3MINDIA，EQ，11279.05,11356.85,11115,11174.25,11150， 11279.8,633,7096578.7,04-DEC-2015,212，INE470A01017，

Spring config file :
   <beans xmlns="http://www.springframework.org/schema/beans"
   xmlns:batch="http://www.springframework.org/schema/batch" 
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:util="http://www.springframework.org/schema/util"
   xsi:schemaLocation="http://www.springframework.org/schema/batch
    http://www.springframework.org/schema/batch/spring-batch-2.2.xsd
    http://www.springframework.org/schema/beans 
    http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
    http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util-3.0.xsd">

<import resource="../config/context.xml" />
<import resource="../config/database.xml" />

<bean id="report" class="com.equities.batch.Report" scope="prototype" />
<bean id="itemProcessor" class="com.equities.batch.CustomItemProcessor" />

<batch:job id="eodBatchJob" restartable="true">
  <batch:step id="step1">
    <batch:tasklet>
        <batch:chunk reader="csvFileItemReader" writer="xmlItemWriter" 
                          processor="itemProcessor" commit-interval="10">
        </batch:chunk>
    </batch:tasklet>
  </batch:step>
</batch:job>

<bean id="csvFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"   scope="step">
            <property name="linesToSkip" value="1"></property>

    <property name="resource" value="classpath:#{jobParameters['FILE']}" />

    <property name="lineMapper">
        <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
        <property name="lineTokenizer">
            <bean
                class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                <property name="delimiter">
                    <util:constant
                        static-field="org.springframework.batch.item.file.transform.DelimitedLineTokenizer.DELIMITER_COMMA" />
                    </property>
                <property name="names" value="SYMBOL,SERIES,OPEN,HIGH,LOW,CLOSE,LAST,PREVCLOSE,TOTTRDQTY,TOTTRDVAL,TIMESTAMP,TOTALTRADES,ISIN" />
            </bean>
        </property>
        <property name="fieldSetMapper">
            <bean class="com.equities.batch.ReportFieldSetMapper" />

             <!-- if no data type conversion, use BeanWrapperFieldSetMapper to map by name
            <bean
                class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
                <property name="prototypeBeanName" value="report" />
            </bean>
             -->
        </property>
        </bean>
    </property>

</bean>

<bean id="xmlItemWriter" class="org.springframework.batch.item.xml.StaxEventItemWriter">
    <property name="resource" value="file:xml/outputs/report.xml" />
    <property name="marshaller" ref="reportMarshaller" />
    <property name="rootTagName" value="report" />
</bean>

<bean id="reportMarshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
   <property name="classesToBeBound">
    <list>
        <value>com.equities.batch.Report</value>
    </list>
    </property>
</bean>

Mapper类：

package com.equities.batch;



import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;

import org.springframework.batch.item.file.mapping.FieldSetMapper;
import org.springframework.batch.item.file.transform.FieldSet;
import org.springframework.validation.BindException;


public class ReportFieldSetMapper implements FieldSetMapper<Report>{

/* (non-Javadoc)
 * @see org.springframework.batch.item.file.mapping.FieldSetMapper#mapFieldSet(org.springframework.batch.item.file.transform.FieldSet)
 */
public Report mapFieldSet(FieldSet fieldSet) throws BindException {

    Report report =new Report();
    System.out.println("Count"+fieldSet.getFieldCount());
    report.setSymbol(fieldSet.readString(0));
    report.setSeries(fieldSet.readString(1));
    report.setOpenPrice(fieldSet.readBigDecimal(2));
    report.setHighPrice(fieldSet.readBigDecimal(3));
    report.setLowPrice(fieldSet.readBigDecimal(4));
    report.setClosePrice(fieldSet.readBigDecimal(5));
    report.setLastTradedPrice(fieldSet.readBigDecimal(6));
    report.setPrevClosePrice(fieldSet.readBigDecimal(7));
    report.setTotalTradedQty(fieldSet.readInt(8));
    report.setTotalTradedValue(fieldSet.readBigDecimal(9));

    String date=fieldSet.readString(10);
    System.out.println("DATE"+date);
    SimpleDateFormat formatter=new SimpleDateFormat("dd-MMM-yy");

    try {
        System.out.println("DATE formatted"+formatter.parse(date));

        report.setDate(formatter.parse(date));
    } catch (ParseException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

    report.setTotalTrades(fieldSet.readInt(11));
    report.setISIN(fieldSet.readString(12));



    return report;
}

}

1 个答案:

答案 0 :(得分：1)

感谢所有的建议，但是当我在names属性中添加额外的'，'时，问题就解决了。

<bean
                class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                <property name="delimiter">
                    <util:constant
                        static-field="org.springframework.batch.item.file.transform.DelimitedLineTokenizer.DELIMITER_COMMA" />
                    </property>
                <property name="names" value="SYMBOL,SERIES,OPEN,HIGH,LOW,CLOSE,LAST,PREVCLOSE,TOTTRDQTY,TOTTRDVAL,TIMESTAMP,TOTALTRADES,ISIN," />
            </bean>
相关问题
最新问题