我正在尝试使用相当简单的路径通过SFTP将文件从远程服务器下载到本地目录。
我面临的问题是该路由能够下载小文件(200字节),但是对于较大的文件(200 KB),它会完全卡在98265字节。大约30秒后,我收到管道封闭错误。
我正在使用Apache Camel版本:2.15.3
发件人:sftp://xxx.xxx.xxx.xxx:xx //用户/用户01?延迟= 30s& include =文件。*。csv& initialDelay = 1m& password = xxxxxx& stepwise = false& streamDownload =真安培;用户名=用户01
收件人:file:/// var / opt / myfolder / incoming?doneFileName = $ {file:name} .done
DEBUG 04/11/15 07:45:24,183 org.apache.camel.component.file.FileOperations :Using InputStream to write file: /var/opt/myfolder/incoming/File01.csv
...
few seconds gap
...
INFO 04/11/15 07:49:53,820 org.apache.camel.component.file.remote.SftpOperations$JSchLogger :JSCH -> Caught an exception, leaving main loop due to Connection reset
INFO 04/11/15 07:49:53,821 org.apache.camel.component.file.remote.SftpOperations$JSchLogger :JSCH -> Disconnecting from xxx.xxx.xxx.xxx port xx
WARN 04/11/15 07:49:53,823 org.apache.camel.util.IOHelper :Cannot close: File01.csv. Reason: Pipe closed
java.io.IOException: Pipe closed
at java.io.PipedInputStream.read(PipedInputStream.java:308)
at java.io.PipedInputStream.read(PipedInputStream.java:378)
at java.io.InputStream.skip(InputStream.java:222)
at com.jcraft.jsch.ChannelSftp.skip(ChannelSftp.java:2894)
at com.jcraft.jsch.ChannelSftp.access$600(ChannelSftp.java:36)
at com.jcraft.jsch.ChannelSftp$RequestQueue.cancel(ChannelSftp.java:1246)
at com.jcraft.jsch.ChannelSftp$2.close(ChannelSftp.java:1503)
at org.apache.camel.util.IOHelper.close(IOHelper.java:326)
at org.apache.camel.component.file.FileOperations.writeFileByStream(FileOperations.java:404)
at org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:274)
at org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:277)
at org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:165)
at org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:79)
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:129)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at org.apache.camel.processor.ChoiceProcessor.process(ChoiceProcessor.java:111)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:60)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:166)
at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:435)
at org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:137)
at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211)
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175)
at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
发生此错误后,将下载此文件并且不会删除任何剩余文件。 我也尝试过使用localWorkDirectory,但是看到tmp目录下载的文件也停留在98265字节。
+更新:
经过进一步调查后,似乎Apache Camel内部使用的JSCH库存在问题。等待时间过后我得到以下错误。我创建了一个使用JSCH进行下载的独立代码,它也悬挂在下载之间。我还用另一个sftp api(不是JSCH)创建了一个独立的代码,它能够成功下载文件。
WARN 04/11/15 07:49:58,877 org.slf4j.helpers.MarkerIgnoringBase :Error processing file RemoteFile[/User/User01/File02.csv] due to Cannot retrieve file: /User/User01/File02.csv. Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot retrieve file: /User/User01/File02.csv]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot retrieve file: /User/User01/File02.csv
at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:651)
at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:594)
at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:396)
at org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:137)
at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211)
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175)
at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: 4:
at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1513)
at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1266)
at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:636)
... 14 more
Caused by: java.io.IOException: Pipe closed
at java.io.PipedInputStream.read(PipedInputStream.java:308)
at com.jcraft.jsch.Channel$MyPipedInputStream.updateReadSide(Channel.java:362)
at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1287)
... 16 more
问题1:可能出现什么问题?如何解决这个问题?
问题2:有没有办法将任何其他库与Apache Camel用于SFTP?
任何建议都将不胜感激。
提前致谢。