oozie运行示例错误:IllegalArgumentException:错误的FS:hdfs:// ** / user / ubuntu / share / lib,expected:file:///

时间:2016-07-21 14:48:39

标签: hadoop oozie

最近,我一直在调查oozie,并努力建立一个本地的oozie系统。在一次又一次阅读官方网页后,我终于成功了。但是当我试图在examples/方向上运行示例时,我总是遇到错误:

2016-07-21 21:55:17,936  WARN ActionStartXCommand:523 - SERVER[namenode] USER[ubuntu] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000001-160720220441466-oozie-ubun-W] ACTION[0000001-160720220441466-oozie-ubun-W@mr-node] Error starting action [mr-node]. ErrorType [ERROR], ErrorCode [IllegalArgumentException], Message [IllegalArgumentException: Wrong FS: hdfs://166.111.81.254:9000/user/ubuntu/share/lib, expected: file:///]
org.apache.oozie.action.ActionExecutorException: IllegalArgumentException: Wrong FS: hdfs://166.111.81.254:9000/user/ubuntu/share/lib, expected: file:///
    at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:445)
    at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1132)
    at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1286)
    at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:250)
    at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:64)
    at org.apache.oozie.command.XCommand.call(XCommand.java:286)
    at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:321)
    at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:250)
    at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Wrong FS: hdfs://166.111.81.254:9000/user/ubuntu/share/lib, expected: file:///
    at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645)
    at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:80)
    at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:372)
    at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1485)
    at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1525)
    at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:570)
    at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1485)
    at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1525)
    at org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:687)
    at org.apache.oozie.service.ShareLibService.updateShareLib(ShareLibService.java:551)
    at org.apache.oozie.service.ShareLibService.getShareLibJars(ShareLibService.java:346)
    at org.apache.oozie.service.ShareLibService.getSystemLibJars(ShareLibService.java:412)
    at org.apache.oozie.action.hadoop.JavaActionExecutor.addSystemShareLibForAction(JavaActionExecutor.java:721)
    at org.apache.oozie.action.hadoop.JavaActionExecutor.addAllShareLibs(JavaActionExecutor.java:818)
    at org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:809)
    at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1037)
    ... 10 more
2016-07-21 21:55:17,939  WARN ActionStartXCommand:523 - SERVER[namenode] USER[ubuntu] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000001-160720220441466-oozie-ubun-W] ACTION[0000001-160720220441466-oozie-ubun-W@mr-node] Setting Action Status to [DONE]

错误困扰了我几天,我一直试图解决它。如果您有任何建议,请教我〜非常感谢!

我的配置是AS FOLLOW:

的hadoop

芯-site.xml中

<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://166.111.81.254:9000</value>
  </property>

  <property>
    <name>hadoop.proxyuser.ubuntu.hosts</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.ubuntu.groups</name>
    <value>*</value>
  </property>
</configuration>

纱-site.xml中

<configuration>

<!-- Site specific YARN configuration properties -->
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>
  <property>
    <name>yarn.resourcemanager.hostname</name>
    <value>166.111.81.254</value>
  </property>
</configuration>

mapred-site.xml中

<configuration>
  <property>
    <name>mapreduce.jobtracker.address</name>
    <value>166.111.81.254:54311</value>
  </property>
  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
</configuration>

HDFS-site.xml中

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>2</value>
  </property>
  <property>
    <name>dfs.namenode.name.dir</name>
    <value>file:///usr/local/hadoop-2.6.4/hadoop_data/hdfs/namenode</value>
  </property>
  <property>
    <name>dfs.permissions.enabled</name>
    <value>false</value>
  </property>
  <property>
    <name>dfs.webhdfs.enabled</name>
    <value>true</value>
  </property>
</configuration>

Oozie的

Oozie的-site.xml中

<configuration>
  <property>
      <name>oozie.service.HadoopAccessorService.root.configurations</name>
      <value>*=/usr/local/hadoop-2.6.4/etc/hadoop/</value>
      <description>
          Comma separated AUTHORITY=HADOOP_CONF_DIR, where AUTHORITY is the HOST:PORT of
          the Hadoop service (JobTracker, HDFS). The wildcard '*' configuration is
          used when there is no exact match for an authority. The HADOOP_CONF_DIR contains
          the relevant Hadoop *-site.xml files. If the path is relative is looked within
          the Oozie configuration directory; though the path can be absolute (i.e. to point
          to Hadoop client conf/ directories in the local filesystem.
      </description>
  </property>
  <property>
    <name>oozie.service.WorkflowAppService.system.libpath</name>
    <value>hdfs://166.111.81.254:9000/user/${user.name}/share/lib</value>
    <description>
        System library path to use for workflow applications.
        This path is added to workflow application if their job properties sets
        the property 'oozie.use.system.libpath' to true.
    </description>
</property>
<property>
    <name>oozie.service.ProxyUserService.proxyuser.ubuntu.hosts</name>
    <value>*</value>
</property>
<property>
    <name>oozie.service.ProxyUserService.proxyuser.ubuntu.groups</name>
    <value>*</value>
</property>

0 个答案:

没有答案