Question

如何在命令行上设置hadoop.tmp.dir？我试图通过将其添加到命令行来添加它 - 当它也不能通过HADOOP_OPTS工作时添加：

export HADOOP_OPTS =＆＃34; -Dhadoop.tmp.dir = / tmp / tmp

该目录是专门为了世界可写而创建的：

$ls -lrta /tmp/tmp
total 0
drwxrwxrwx   2 myuser   wheel    64 Apr 27 01:23 .
drwxrwxrwt  45 root     wheel  1440 Apr 27 01:23 ..

但hadoop继续使用它想要的任何默认tmpdir：

$hadoop jar target/mp5-1.0-SNAPSHOT-jar-with-dependencies.jar org.apache.giraph.GiraphRunner -Dmapreduce.child.java.opts="-Dhadoop.tmpdir=/tmp/tmp -Djava.io.tmpdir=/tp/tmp" ConnectedComponentsComputation -vif org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip dataset/graph.data -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op file://$(pwd)/output/part-a -w 1 -ca giraph.SplitMasterWorker=false

但总是我们最终会遇到权限错误;

Exception in thread "main" java.io.IOException: 
Mkdirs failed to create /var/folders/3t/4d3czfm17fb2v9z_m8rvflyrg8jmdl/T/hadoop-unjar2119053878070751096/META-INF/license
    at org.apache.hadoop.util.RunJar.ensureDirectory(RunJar.java:140)
    at org.apache.hadoop.util.RunJar.unJar(RunJar.java:109)
    at org.apache.hadoop.util.RunJar.unJar(RunJar.java:85)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:222)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:148)

那么如何在命令行上配置hadoop或者至少通过env vars配置（不是通过更改hadoop配置文件本身）？

（顺便说一句还设置$HADOOP_CONF_DIR/core-site.xml .. 仍然没有骰子）

$grep  -C 2 hadoop.tmp.dir $HADOOP_CONF_DIR/core-site.xml
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/tmp/tmp</value>
    </property>

hadoop.tmp.dir没有被hadoop命令所尊重

0 个答案: