错误:将文件放入HDFS

时间:2013-10-08 15:34:06

标签: hadoop hdfs

尝试在ARM处理器上运行示例Hadoop源代码时遇到问题。每次我试图将一些文件放入HDFS时,我都会收到以下错误。

13/10/07 11:31:29 WARN hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/root/bin/cpu-kmeans2D could only be replicated to 0 nodes, instead of 1
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)

    at org.apache.hadoop.ipc.Client.call(Client.java:739)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
    at com.sun.proxy.$Proxy0.addBlock(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
    at com.sun.proxy.$Proxy0.addBlock(Unknown Source)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2904)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2786)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.java:2076)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2262)

13/10/07 11:31:29 WARN hdfs.DFSClient: Error Recovery for block null bad datanode[0] nodes == null
13/10/07 11:31:29 WARN hdfs.DFSClient: Could not get block locations. Source file "/user/root/bin/cpu-kmeans2D" - Aborting...
put: java.io.IOException: File /user/root/bin/cpu-kmeans2D could only be replicated to 0 nodes, instead of 1

我尝试通过删除主节点和从节点上的所有旧日志以及/app/hadoop/下的文件夹来复制namenode和datanode,之后我格式化了namenode并再次启动了该进程({{1但是,但仍然没有运气。

我在重新启动后尝试生成管理报告(粘贴在下面),似乎数据节点没有开始。

bin/start-all.sh

我已经尝试了所有可能的方法来调试它。我尝试了以下方法:

1)我登录HADOOP主目录并删除了所有旧日志(------------------------------------------------- Datanodes available: 0 (0 total, 0 dead) root@tegra-ubuntu:~/hadoop-gpu-master/hadoop-gpu-0.20.1# bin/hadoop dfsadmin -report Configured Capacity: 0 (0 KB) Present Capacity: 0 (0 KB) DFS Remaining: 0 (0 KB) DFS Used: 0 (0 KB) DFS Used%: �% Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 ------------------------------------------------- Datanodes available: 0 (0 total, 0 dead)

2)接下来我删除了所有从节点和主节点上的目录内容(rm -rf logs/*

3)我格式化了namenode(rm -rf /app/hadoop/*

4)我开始了所有的过程 - 首先是namenode,datanode,然后是map - reduce。我在终端上输入了jps,以确保所有进程(Namenode,Datanode,JobTracker,Task Tracker)都已启动并运行。

5)现在这样做,我重新创建了dfs中的目录。

0 个答案:

没有答案