如何使用JAVA API在HDFS中移动或复制文件

Question

我想在相同的HDFS中复制文件，就像从HDFS复制文件：// abc：9000 / user / a.txt到HDFS：// abc：9000 / user / 123 /

我可以使用JAVA API吗？感谢

Answer 1

FileUtil提供了复制文件的方法。

Configuration configuration = new Configuration();
configuration.set("fs.defaultFS", "hdfs://abc:9000");
FileSystem filesystem = FileSystem.get(configuration);
FileUtil.copy(filesystem, new Path("src/path"), filesystem, new Path("dst/path"), false, configuration);

如果您需要将其复制到其他群集，只需制作新的Configuration并设置新的FileSystem。

Answer 2

If you want to move files from directory it is little bit tricky below code done same task for me !!
val conf = new org.apache.hadoop.conf.Configuration()
    val src:Path = new org.apache.hadoop.fs.Path(hdfsDirectory)
    val fs = FileSystem.get(src.toUri,conf)
    val srcPath: Path = new Path("hdfs://sourcePath/")
    val srcFs =FileSystem.get(srcPath.toUri,conf)
    val dstPath:Path =new Path("hdfs://targetPath/")
    val dstFs =FileSystem.get(dstPath.toUri,conf)
    val exists = fs.exists(new org.apache.hadoop.fs.Path(hdfsDirectory))
    val status:Array[FileStatus] = fs.listStatus(new Path(hdfsDirectory))
    if (status.length>0) {
      status.foreach(x => {
        println("My files: " + x.getPath)
        FileUtil.copy(srcFs, x.getPath, dstFs, dstPath, true, conf)
        println("Files moved !!" +x.getPath)
      }
      )}
    else{
      println("No Files Found !!")
    }