Question

我收到以下关于找不到文件的错误。嗯......文件存在。我是distcp的新手。我正在使用cloudera FYI。

 https://s3.amazonaws.com/test-development/test/201305031003_0_ubuntu.gz


ubuntu@ubuntu:~$ hadoop distcp -i 201305031003_0_ubuntu.gz s3://id:key@test-development/test/201305031003_0_ubuntu.gz
13/05/04 14:54:29 INFO tools.DistCp: srcPaths=[201305031003_0_ubuntu.gz]
13/05/04 14:54:29 INFO tools.DistCp: destPath=s3://id:key@test-development/test/201305031003_0_ubuntu.gz
With failures, global counters are inaccurate; consider running with -i
Copy failed: org.apache.hadoop.mapred.InvalidInputException: Input source 201305031003_0_ubuntu.gz does not exist.
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:641)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:656)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)

Answer 1

第一个参数是源，所以它应该是S3的路径，路径应该是s3n：//而不是s3：//（本机s3），除非你使用s3：//将数据写入S3 （块文件系统）

从s3到hadoop的distcp - 找不到文件

1 个答案: