Question

我在使用Windows7企业版的本地PC上使用springXD 1.3.1，并在Microsoft Azure云上配置了Hortonworks。在Azure的Hortonworks上，我创建了一个dir xd并提供了springxd doc中提到的所需权限。然后我在config / server.yml文件中输入了以下条目：

spring:
  profiles: singlenode
  hadoop:
   fsUri: hdfs://13.92.199.104:8020
   resourceManagerHost: 13.92.199.104
   resourceManagerPort: 8050

此外，我在config / hadoop.properties文件中输入了

fs.default.name=hdfs://13.92.199.104:8020

然后，我通过像xd-singlenode.bat之类的命令启动springxd，然后通过xd-shell.bat启动shell

现在在shell控制台上我运行像

这样的命令

hadoop config fs --namenode hdfs://13.92.199.104:8020

直到现在，一切正常。但是，如果现在我运行hadoop fs ls /xd之类的命令，或者如果我尝试部署像time | hdfs --fsUri=hdfs://13.92.199.104/这样的简单流，那么会出现以下错误：

1.3.1.RELEASE | Admin Server Target: http://localhost:9393
Welcome to the Spring XD shell. For assistance hit TAB or type "help".
xd:>hadoop config fs --namenode hdfs://13.92.199.104:8020
xd:>hadoop fs ls /xd
2016-04-15T16:41:10+0530 ERROR Spring Shell o.a.h.u.Shell:374 - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
        at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:356) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:371) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.util.Shell.<clinit>(Shell.java:364) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.fs.FsShellPermissions.<clinit>(FsShellPermissions.java:114) [hadoop-common-2.7.1.jar:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77]
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_77]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_77]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_77]
        at org.apache.hadoop.fs.shell.CommandFactory.registerCommands(CommandFactory.java:68) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.fs.shell.FsCommand.registerCommands(FsCommand.java:53) [hadoop-common-2.7.1.jar:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77]
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_77]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_77]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_77]
        at org.apache.hadoop.fs.shell.CommandFactory.registerCommands(CommandFactory.java:68) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.fs.FsShell.registerCommands(FsShell.java:101) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.fs.FsShell.init(FsShell.java:92) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.fs.FsShell.run(FsShell.java:274) [hadoop-common-2.7.1.jar:na]
        at org.springframework.xd.shell.hadoop.FsShellCommands.run(FsShellCommands.java:418) [spring-xd-shell-1.3.1.RELEASE.jar:1.3.1.RELEASE]
        at org.springframework.xd.shell.hadoop.FsShellCommands.runCommand(FsShellCommands.java:413) [spring-xd-shell-1.3.1.RELEASE.jar:1.3.1.RELEASE]
        at org.springframework.xd.shell.hadoop.FsShellCommands.ls(FsShellCommands.java:116) [spring-xd-shell-1.3.1.RELEASE.jar:1.3.1.RELEASE]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77]
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_77]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_77]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_77]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:216) [spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
        at org.springframework.shell.core.SimpleExecutionStrategy.invoke(SimpleExecutionStrategy.java:64) [spring-shell-1.1.0.RELEASE.jar:1.1.0.RELEASE]
        at org.springframework.shell.core.SimpleExecutionStrategy.execute(SimpleExecutionStrategy.java:48) [spring-shell-1.1.0.RELEASE.jar:1.1.0.RELEASE]
        at org.springframework.shell.core.AbstractShell.executeCommand(AbstractShell.java:127) [spring-shell-1.1.0.RELEASE.jar:1.1.0.RELEASE]
        at org.springframework.shell.core.JLineShell.promptLoop(JLineShell.java:533) [spring-shell-1.1.0.RELEASE.jar:1.1.0.RELEASE]
        at org.springframework.shell.core.JLineShell.run(JLineShell.java:179) [spring-shell-1.1.0.RELEASE.jar:1.1.0.RELEASE]
        at java.lang.Thread.run(Unknown Source) [na:1.8.0_77]
Found 3 items
drwxrwxrwx   - jitendra.kumar.singh hdfs          0 2016-04-15 15:42 /xd/asdsadasdsad
drwxrwxrwx   - jitendra.kumar.singh hdfs          0 2016-04-15 14:30 /xd/fsd
drwxrwxrwx   - jitendra.kumar.singh hdfs          0 2016-04-15 14:34 /xd/timeLogHdfs

是Windows特定错误吗？我怎么解决它。请提出一些解决此问题的方法。

Answer 1

尝试将winutils.exe下载到Hadoop“home”目录中，并设置指向此目录的系统属性。有关详细信息，请参阅https://github.com/spring-projects/spring-hadoop/wiki/Using-a-Windows-client-together-with-a-Linux-cluster。不确定这会解决您的问题，但值得一试。

使用Hortonwork在spirngxd中运行流时获取错误winutils.exe

1 个答案: