如何在HDFS中创建所需大小的空文件？

Question

我是Hadoop和HDFS的新手。我相信我的问题与this post有些相关。本质上，我试图在HDFS中创建10 GB大小的空文件。 truncate命令失败，因为指定的文件大小似乎被禁止大于现有文件大小。在这种情况下，有哪些替代方案？例如，在Linux系统中，可以使用＆＃34; truncate＆＃34;命令设置任意文件大小。

Answer 1

You can use TestDFSIO to create the file with the required size in HDFS directly.

Program TestDFSIO is packaged in jar file 'hadoop-mapreduce-client-jobclient-tests.jar'. This jar comes with the hadoop installation, locate this jar and provide the path of this jar in the below command.

hadoop jar <PATH_OF_JAR_hadoop-mapreduce-client-jobclient-tests.jar> TestDFSIO -write -nrFiles 1 -fileSize 10GB

where "nrFiles" is Number of files and "filesize" is each file size to be generated.

File will be generated at path /benchmarks/TestDFSIO/ in HDFS.