Hadoop put和copyfromlocal之间的区别

时间:2018-09-12 11:16:54

标签: hadoop hdfs

CopyFromLocal将从本地文件系统上载数据。

Do put将从任何文件上传数据,例如。本地FS,亚马逊S3 或仅来自本地fs ???

2 个答案:

答案 0 :(得分:0)

请找到两个命令的用法。

put
=======
Usage: hadoop fs -put [-f] [-p] [-l] [-d] [ - | <localsrc1> .. ]. <dst>

Copy single src, or multiple srcs from local file system to the destination file system. Also reads input from stdin and writes to destination file system if the source is set to “-”

Copying fails if the file already exists, unless the -f flag is given.

Options:

-p : Preserves access and modification times, ownership and the permissions. (assuming the permissions can be propagated across filesystems)
-f : Overwrites the destination if it already exists.
-l : Allow DataNode to lazily persist the file to disk, Forces a replication factor of 1. This flag will result in reduced durability. Use with care.
-d : Skip creation of temporary file with the suffix ._COPYING_.
Examples:

hadoop fs -put localfile /user/hadoop/hadoopfile
hadoop fs -put -f localfile1 localfile2 /user/hadoop/hadoopdir
hadoop fs -put -d localfile hdfs://nn.example.com/hadoop/hadoopfile
hadoop fs -put - hdfs://nn.example.com/hadoop/hadoopfile Reads the input from stdin.
Exit Code:

Returns 0 on success and -1 on error.

copyFromLocal
=============

Usage: hadoop fs -copyFromLocal <localsrc> URI

Similar to the fs -put command, except that the source is restricted to a local file reference.

Options:

-p : Preserves access and modification times, ownership and the permissions. (assuming the permissions can be propagated across filesystems)
-f : Overwrites the destination if it already exists.
-l : Allow DataNode to lazily persist the file to disk, Forces a replication factor of 1. This flag will result in reduced durability. Use with care.
-d : Skip creation of temporary file with the suffix ._COPYING_.

答案 1 :(得分:0)

它们是identical(按照extends Put,没有其他方法或附加条件)。截至该答案为止,该CopyCommands.java类自2014年以来就没有新的提交。

  public static class CopyFromLocal extends Put {
        public static final String NAME = "copyFromLocal";
        public static final String USAGE = Put.USAGE;
        public static final String DESCRIPTION = "Identical to the -put command.";
  }
  

请提供使用“ hadoop fs-put”命令可以获取/输入数据的所有文件系统名称

使用hadoop fs命令支持任何Hadoop Compatible FileSystem (HCFS)hdfs dfs

仅支持HDFS