回收槽无法完成向s3的传输

时间:2019-02-01 15:32:33

标签: amazon-s3 apache-flink

当数据是越来越使用铲装水槽写入S3。我在例外之下。只是想知道,如果做任何事情需要。

PS:但该文件得到从-in-进展移动到最终状态

  

TimerException {org.apache.hadoop.fs.s3a.AWSClientIOException:正在保存   输出   home / prod / dt = 20190201 / hr = 12 / _part-1-7。进行中:   com.amazonaws.AmazonClientException:无法完全转移:   com / amazonaws / util / Throwables:无法完成转移:   COM / amazonaws / UTIL /将Throwable}在   org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService $ TriggerTask.run(SystemProcessingTimeService.java:288)     在   java.util.concurrent.Executors $ RunnableAdapter.call(Executors.java:511)     在java.util.concurrent.FutureTask.run(FutureTask.java:266)在   java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.access $ 201(ScheduledThreadPoolExecutor.java:180)     在   java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)     在   java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)     在   java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:624)     在java.lang.Thread.run(Thread.java:748)造成原因:   org.apache.hadoop.fs.s3a.AWSClientIOException:将输出保存在   home / prod / dt = 20190201 / hr = 12 / _part-1-7。进行中:   com.amazonaws.AmazonClientException:无法完成转移:   com / amazonaws / util / Throwables:无法完成转移:   COM / amazonaws / UTIL /将Throwable在   org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:128)     在   org.apache.hadoop.fs.s3a.S3AOutputStream.close(S3AOutputStream.java:121)     在   org.apache.hadoop.fs.FSDataOutputStream $ PositionCache.close(FSDataOutputStream.java:72)     在   org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)     在   org.apache.flink.streaming.connectors.fs.StreamWriterBase.close(StreamWriterBase.java:99)     在   org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.closeCurrentPartFile(BucketingSink.java:592)     在   org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.closePartFilesByTime(BucketingSink.java:525)     在   org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.onProcessingTime(BucketingSink.java:506)     在   org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService $ TriggerTask.run(SystemProcessingTimeService.java:285)     ... 7更多原因:com.amazonaws.AmazonClientException:无法执行   完全转移:COM / amazonaws / UTIL /将Throwable在   com.amazonaws.services.s3.transfer.internal.AbstractTransfer.unwrapExecutionException(AbstractTransfer.java:277)     在   com.amazonaws.services.s3.transfer.internal.AbstractTransfer.rethrowExecutionException(AbstractTransfer.java:261)     在   com.amazonaws.services.s3.transfer.internal.UploadImpl.waitForUploadResult(UploadImpl.java:66)在   org.apache.hadoop.fs.s3a.S3AOutputStream.close(S3AOutputStream.java:113)     java.lang.NoClassDefFoundError:... 14多产生的原因:   COM / amazonaws / UTIL /将Throwable在   com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1475)     在   com.amazonaws.services.s3.transfer.internal.UploadCallable.uploadInOneChunk(UploadCallable.java:131)     在   com.amazonaws.services.s3.transfer.internal.UploadCallable.call(UploadCallable.java:123)     在   com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:139)     在   com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:47)     在java.util.concurrent.FutureTask.run(FutureTask.java:266)... 3   更多

0 个答案:

没有答案