PDI在读取远程和压缩文件时失败

时间：2017-12-13 17:07:02

标签： pentaho pentaho-data-integration

我的问题是当我从S3读取一个远程压缩文件时，这是一个特定的zip文件。该文件没有损坏，有时会获得异常，有时甚至没有。

我在这个仍然打开的旧版本http://jira.pentaho.com/browse/PDI-1800

中发现了类似的问题

这是发生的错误。我在一个carte服务器上运行这个转换。这个错误不容易重现，不幸的是，我没有配方来重现它。

org.pentaho.di.core.exception.KettleFileException:
Exception reading line: java.io.EOFException: Unexpected end of ZLIB input stream
Unexpected end of ZLIB input stream
Unexpected end of ZLIB input stream
at org.pentaho.di.trans.steps.fileinput.text.TextFileInputUtils.getLine(TextFileInputUtils.java:326)
at org.pentaho.di.trans.steps.fileinput.text.TextFileInputReader.tryToReadLine(TextFileInputReader.java:420)
at org.pentaho.di.trans.steps.fileinput.text.TextFileInputReader.readRow(TextFileInputReader.java:167)
at org.pentaho.di.trans.steps.fileinput.BaseFileInputStep.processRow(BaseFileInputStep.java:205)
at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.EOFException: Unexpected end of ZLIB input stream
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at java.util.zip.ZipInputStream.read(ZipInputStream.java:194)
at org.pentaho.di.core.compress.CompressionInputStream.read(CompressionInputStream.java:68)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at sun.nio.cs.StreamDecoder.read0(StreamDecoder.java:127)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:112)
at java.io.InputStreamReader.read(InputStreamReader.java:168)
at org.pentaho.di.trans.steps.fileinput.text.TextFileInputUtils.getLine(TextFileInputUtils.java:294)
… 5 more
2017/12/13 12:07:40 – S3CsvInput.0 – ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : Unexpected error
2017/12/13 12:07:40 – S3CsvInput.0 – ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : org.pentaho.di.core.exception.KettleFileException:
2017/12/13 12:07:40 – S3CsvInput.0 –
2017/12/13 12:07:40 – S3CsvInput.0 – Exception reading line: java.io.EOFException: Unexpected end of ZLIB input stream
2017/12/13 12:07:40 – S3CsvInput.0 – Unexpected end of ZLIB input stream
2017/12/13 12:07:40 – S3CsvInput.0 –
2017/12/13 12:07:40 – S3CsvInput.0 – Unexpected end of ZLIB input stream
2017/12/13 12:07:40 – S3CsvInput.0 –
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.trans.steps.fileinput.text.TextFileInputUtils.getLine(TextFileInputUtils.java:326)
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.trans.steps.fileinput.text.TextFileInputReader.tryToReadLine(TextFileInputReader.java:420)
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.trans.steps.fileinput.text.TextFileInputReader.readRow(TextFileInputReader.java:167)
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.trans.steps.fileinput.BaseFileInputStep.processRow(BaseFileInputStep.java:205)
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2017/12/13 12:07:40 – S3CsvInput.0 – at java.lang.Thread.run(Thread.java:745)
2017/12/13 12:07:40 – S3CsvInput.0 – Caused by: java.io.EOFException: Unexpected end of ZLIB input stream
2017/12/13 12:07:40 – S3CsvInput.0 – at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
2017/12/13 12:07:40 – S3CsvInput.0 – at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
2017/12/13 12:07:40 – S3CsvInput.0 – at java.util.zip.ZipInputStream.read(ZipInputStream.java:194)
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.core.compress.CompressionInputStream.read(CompressionInputStream.java:68)
2017/12/13 12:07:40 – S3CsvInput.0 – at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
2017/12/13 12:07:40 – S3CsvInput.0 – at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
2017/12/13 12:07:40 – S3CsvInput.0 – at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
2017/12/13 12:07:40 – S3CsvInput.0 – at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
2017/12/13 12:07:40 – S3CsvInput.0 – at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
2017/12/13 12:07:40 – S3CsvInput.0 – at sun.nio.cs.StreamDecoder.read0(StreamDecoder.java:127)
2017/12/13 12:07:40 – S3CsvInput.0 – at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:112)
2017/12/13 12:07:40 – S3CsvInput.0 – at java.io.InputStreamReader.read(InputStreamReader.java:168)
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.trans.steps.fileinput.text.TextFileInputUtils.getLine(TextFileInputUtils.java:294)
2017/12/13 12:07:40 – S3CsvInput.0 – … 5 more
child index = 56, logging object : org.pentaho.di.core.logging.LoggingObject@46345a23 parent=1ff32099-5cbe-47b3-b32c-34f1291f6c09
2017/12/13 12:07:40 – md5_field12308.0 – Finished processing (I=0, O=0, R=391170, W=782340, U=0, E=0)
2017/12/13 12:07:40 – Filter Rows field12308.0 – Finished processing (I=0, O=0, R=3, W=3, U=0, E=0)
2017/12/13 12:07:40 – Filter Rows field12315.0 – Finished processing (I=0, O=0, R=13, W=13, U=0, E=0)
2017/12/13 12:07:40 – TransLoad_cube5403_data137170 – ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : Errors detected!

1 个答案:

答案 0 :(得分：0)

很可能是一个网络问题;该文件未完全下载，因此无法正常终止。

首先尝试下载并从本地文件中读取，然后查看问题是否仍然存在。

相关问题
最新问题