Question

我的hadoop版本为2.7.1，我的oozie版本为3.3.2。我已成功创建了oozie.war文件。但是，当我尝试创建共享库时hdfs使用以下命令

oozie-setup.sh sharelib create -fs hdfs：// training-21：54310

我得到以下异常，

Stack trace for the error was (for debug purposes):
--------------------------------------
org.apache.oozie.service.HadoopAccessorException: E0902: Exception occured: [Server IPC version 9 cannot communicate with client version 4]
    at org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:433)
    at org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:144)
    at org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:52)
Caused by: org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4
    at org.apache.hadoop.ipc.Client.call(Client.java:1107)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
    at com.sun.proxy.$Proxy3.getProtocolVersion(Unknown Source)
    at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:411)
    at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:135)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:276)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:241)
    at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:100)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1411)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1429)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
    at org.apache.oozie.service.HadoopAccessorService$2.run(HadoopAccessorService.java:425)
    at org.apache.oozie.service.HadoopAccessorService$2.run(HadoopAccessorService.java:423)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
    at org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:423)
    ... 2 more
-------------------

我只修改了以下配置文件：

芯的site.xml（hadoop的）

<configuration>
 <property>
    <name>hadoop.tmp.dir</name>
    <value>/home/user1/tmp</value>
</property>

     <property>
         <name>fs.default.name</name>
         <value>hdfs://TRAINING-21:54310</value>
     </property>
   <property>
     <name>hadoop.proxyuser.user1.hosts</name>
     <value>*</value>
   </property>

   <property>
     <name>hadoop.proxyuser.user1.groups</name>
     <value>*</value>
   </property>
</configuration>

我尝试使用不同版本的oozie（4.2.0），但问题仍然存在。我该怎么办才能解决此异常？

Answer 1

默认情况下，Oozie针对Hadoop版本0.23.5构建，因此要针对Hadoop 2.7.1构建，我们必须在pom.xml中配置maven依赖项

在下载的Oozie源代码（pom.xml）中，hadoop-2 maven配置文件指定hadoop.version＆amp; hadoop.auth.version为2.3.0。所以我们需要改变他们使用2.7.1。

修改hadoop-2配置文件，如下所示

<profile>
        <id>hadoop-2</id>
        <activation>
            <activeByDefault>false</activeByDefault>
        </activation>
        <properties>
           <hadoop.version>2.7.1</hadoop.version>
           <hadoop.auth.version>2.7.1</hadoop.auth.version>
           <pig.classifier>h2</pig.classifier>
           <sqoop.classifier>hadoop200</sqoop.classifier>
        </properties>
    </profile>

转到hadooplibs

cd hadooplibs

转到hadoop-2

cd hadoop-2

按如下方式编辑pom.xml

<parent>
    <groupId>org.apache.oozie</groupId>
    <artifactId>oozie-main</artifactId>
    <version>4.1.0</version>
    <relativePath>../../pom.xml</relativePath>
</parent>
<groupId>org.apache.oozie</groupId>
<artifactId>oozie-hadoop</artifactId>
<version>2.7.1.oozie-4.1.0</version>
<description>Apache Oozie Hadoop ${project.version}</description>
<name>Apache Oozie Hadoop ${project.version}</name>
<packaging>jar</packaging>

<dependencies>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-client</artifactId>
        <version>2.7.1</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-auth</artifactId>
        <version>2.7.1</version>
    </dependency>

转到hadoop-distcp-2

cd ../ hadoop-distcp-2 /

按如下方式编辑pom.xml。

<artifactId>oozie-hadoop-distcp</artifactId>
<version>2.7.1.oozie-4.1.0</version>
<description>Apache Oozie Hadoop Distcp ${project.version}</description>
<name>Apache Oozie Hadoop Distcp ${project.version}</name>
<packaging>jar</packaging>

<dependencies>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-distcp</artifactId>
        <version>2.7.1</version>
        <scope>compile</scope>
    </dependency>

转到hadoop-test-2

cd ../ hadoop-test-2

按如下方式编辑pom.xml。

<artifactId>oozie-hadoop-test</artifactId>
<version>2.7.1.oozie-4.1.0</version>
<description>Apache Oozie Hadoop ${project.version} Test</description>
<name>Apache Oozie Hadoop ${project.version} Test</name>
<packaging>jar</packaging>

<dependencies>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-minicluster</artifactId>
        <version>2.7.1</version>
        <scope>compile</scope>
    </dependency>

使用hadoop 2.7.1构建Oozie，如下所示。

bin/mkdistro.sh ‐P hadoop‐2 ‐DskipTests

然后按照常规程序进行操作。

获取E0902：发生异常：[服务器IPC版本9无法与客户端版本4通信]

1 个答案: