如何以zip文件格式并行处理文件?

时间:2013-12-10 06:28:41

标签: scala apache-camel

作为我们为客户构建的适配器服务的一部分,我们需要处理来自第三方服务的zip文件中收到的XML,并将其以JSON格式发布到另一个API。除非我们为XML引入并行处理,否则一切正常。这是我们用来读取zip中的XML文件的代码。

this.basicRoute
  .log("picked file fileName=${file:absolute.path}")
  .choice(when(ex => ex.getIn.getHeader(Exchange.FILE_NAME_ONLY).toString.matches(".*\\.zip"))
    .log("inside zip file processing zipFileName=${file:absolute.path}")
    .unmarshal(zipFile)
    .split(_.getIn.getBody)
      .streaming()
      .when(_.getIn.getHeader(Exchange.FILE_NAME).toString.matches(".*\\.xml"))
      .setBody(ex => InputStreamToXmlConverter.toXml(ex.getIn.getBody.asInstanceOf[InputStream]))
      .choice(when(exchange => XmlValidator.isProperLocale(exchange.getIn.getBody.asInstanceOf[Node]))
        .to("direct:toPath"))
      .otherwise()
        .to("file:" + Settings("some.ignoredPath")))

我们尝试了几种并行处理文件的方法,但获得了“Stream Closed”异常。

java.io.IOException: Stream closed
at java.util.zip.ZipInputStream.ensureOpen(ZipInputStream.java:66) ~[na:1.7.0_06]
at java.util.zip.ZipInputStream.read(ZipInputStream.java:181) ~[na:1.7.0_06]
at java.io.BufferedInputStream.read1(BufferedInputStream.java:273) ~[na:1.7.0_06]
at java.io.BufferedInputStream.read(BufferedInputStream.java:334) ~[na:1.7.0_06]
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) ~[na:1.7.0_06]
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) ~[na:1.7.0_06]
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) ~[na:1.7.0_06]
at java.io.InputStreamReader.read(InputStreamReader.java:184) ~[na:1.7.0_06]
at java.io.BufferedReader.fill(BufferedReader.java:154) ~[na:1.7.0_06]
at java.io.BufferedReader.read(BufferedReader.java:175) ~[na:1.7.0_06]
at scala.io.BufferedSource$$anonfun$iter$1$$anonfun$apply$mcI$sp$1.apply$mcI$sp(BufferedSource.scala:38) ~[scala-library.jar:na]
at scala.io.Codec.wrap(Codec.scala:68) ~[scala-library.jar:na]
at scala.io.BufferedSource$$anonfun$iter$1.apply(BufferedSource.scala:38) ~[scala-library.jar:na]
at scala.io.BufferedSource$$anonfun$iter$1.apply(BufferedSource.scala:38) ~[scala-library.jar:na]
at scala.collection.Iterator$$anon$9.next(Iterator.scala:162) ~[scala-library.jar:0.12.1]
at scala.collection.Iterator$$anon$17.hasNext(Iterator.scala:511) ~[scala-library.jar:0.12.1]
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327) ~[scala-library.jar:0.12.1]
at scala.io.Source.hasNext(Source.scala:226) ~[scala-library.jar:na]
at scala.collection.Iterator$class.foreach(Iterator.scala:727) ~[scala-library.jar:0.12.1]
at scala.io.Source.foreach(Source.scala:178) ~[scala-library.jar:na]
at scala.collection.TraversableOnce$class.addString(TraversableOnce.scala:320) ~[scala-library.jar:0.12.1]
at scala.io.Source.addString(Source.scala:178) ~[scala-library.jar:na]
at scala.collection.TraversableOnce$class.mkString(TraversableOnce.scala:286) ~[scala-library.jar:0.12.1]
at scala.io.Source.mkString(Source.scala:178) ~[scala-library.jar:na]
at scala.collection.TraversableOnce$class.mkString(TraversableOnce.scala:288) ~[scala-library.jar:0.12.1]
at scala.io.Source.mkString(Source.scala:178) ~[scala-library.jar:na]
at scala.collection.TraversableOnce$class.mkString(TraversableOnce.scala:290) ~[scala-library.jar:0.12.1]
at scala.io.Source.mkString(Source.scala:178) ~[scala-library.jar:na]

任何有关如何处理此问题的帮助将不胜感激,谢谢!

1 个答案:

答案 0 :(得分:0)

我对Scala DSL一点也不流利,但不是在线遗漏的结尾括号:

.choice(when(ex => ex.getIn.getHeader(Exchange.FILE_NAME_ONLY).toString.matches(".*\\.zip"))