我正在尝试使用Java代码以编程方式获取给定路径下的所有文件
public static List<String> listFilesFromHDFSPath(Configuration hadoopConfiguration, String hdfsPath)
throws IOException, IllegalArgumentException {
List<String> filePaths = new ArrayList<String>();
Path path = new Path(hdfsPath);
FileSystem fs = path.getFileSystem(hadoopConfiguration);
if(fs.isDirectory(path)){
FileStatus[] fileStatuses = fs.listStatus(path);
for(FileStatus fileStatus : fileStatuses){
if(fileStatus.isFile())
filePaths.add(fileStatus.getPath().toString());
}
}
else {
filePaths.add(path.toString());
}
fs.close();
return filePaths;
}
但是这种方法现在对我有用,该过程在下一行之后消失:
FileSystem fs = path.getFileSystem(hadoopConfiguration);
所以我试图像在Scala中那样,在Java中获得等效的“ hdfs dfs -ls -R”命令:
import scala.sys.process._
val lsResult = Seq("hdfs","dfs","-ls","-R","hdfs://path/demo/").!!
java是否有类似的东西?