在测试环境中使用Hadoop和Zookeeper请求Apache Accumulo错误帮助

时间:2011-12-18 17:58:51

标签: hadoop apache-zookeeper

我正在使用apache accumulo ver 1.4.0,hadoop ver 0.20.2和zookeeper ver 3.3.3建立测试环境。请参阅下面的问题。

Hadoop和Zookeeper一起工作很好,但是当我使用apache孵化器上的程序启动accumulo时,我得到以下zookeeper信息流并发出警告:

2011-12-08 20:13:56,601 - INFO  [main:QuorumPeerConfig@90] - Reading configuration from: /home/hadoop/zookeeper-3.3.3/bin/../conf/zoo.cfg
2011-12-08 20:13:56,603 - WARN  [main:QuorumPeerMain@105] - Either no config or no quorum defined in config, running  in standalone mode
2011-12-08 20:13:56,616 - INFO  [main:QuorumPeerConfig@90] - Reading configuration from: /home/hadoop/zookeeper-3.3.3/bin/../conf/zoo.cfg
2011-12-08 20:13:56,617 - INFO  [main:ZooKeeperServerMain@94] - Starting server
2011-12-08 20:13:56,626 - INFO  [main:Environment@97] - Server environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT
2011-12-08 20:13:56,627 - INFO  [main:Environment@97] - Server environment:host.name.paz
2011-12-08 20:13:56,627 - INFO  [main:Environment@97] - Server environment:java.version=1.6.0_26
2011-12-08 20:13:56,628 - INFO  [main:Environment@97] - Server environment:java.vendor=Sun Microsystems Inc.
2011-12-08 20:13:56,629 - INFO  [main:Environment@97] - Server environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.26/jre
2011-12-08 20:13:56,629 - INFO  [main:Environment@97] - Server environment:java.class.path=/home/hadoop/zookeeper-3.3.3/bin/../build/classes:/home/hadoop/zookeeper-3.3.
3/bin/../build/lib/*.jar:/home/hadoop/zookeeper-3.3.3/bin/../zookeeper-3.3.3.jar:/home/hadoop/zookeeper-3.3.3/bin/../lib/log4j-1.2.15.jar:/home/hadoop/zookeeper-3.3.3/b
in/../lib/jline-0.9.94.jar:/home/hadoop/zookeeper-3.3.3/bin/../src/java/lib/*.jar:/home/hadoop/zookeeper-3.3.3/bin/../conf:
2011-12-08 20:13:56,630 - INFO  [main:Environment@97] - Server environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.26/jre/lib/i386/server:/usr/lib/jvm/java-6-su
n-1.6.0.26/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.26/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2011-12-08 20:13:56,630 - INFO  [main:Environment@97] - Server environment:java.io.tmpdir=/tmp
2011-12-08 20:13:56,631 - INFO  [main:Environment@97] - Server environment:java.compiler=<NA>
2011-12-08 20:13:56,631 - INFO  [main:Environment@97] - Server environment:os.name=Linux
2011-12-08 20:13:56,632 - INFO  [main:Environment@97] - Server environment:os.arch=i386
2011-12-08 20:13:56,633 - INFO  [main:Environment@97] - Server environment:os.version=3.0.0-13-generic
2011-12-08 20:13:56,633 - INFO  [main:Environment@97] - Server environment:user.name=hadoop
2011-12-08 20:13:56,634 - INFO  [main:Environment@97] - Server environment:user.home=/home/hadoop
2011-12-08 20:13:56,634 - INFO  [main:Environment@97] - Server environment:user.dir=/home/hadoop
2011-12-08 20:13:56,641 - INFO  [main:ZooKeeperServer@663] - tickTime set to 2000
2011-12-08 20:13:56,641 - INFO  [main:ZooKeeperServer@672] - minSessionTimeout set to -1
2011-12-08 20:13:56,642 - INFO  [main:ZooKeeperServer@681] - maxSessionTimeout set to -1
2011-12-08 20:13:56,661 - INFO  [main:NIOServerCnxn$Factory@143] - binding to port 0.0.0.0/0.0.0.0:2181
2011-12-08 20:13:56,691 - INFO  [main:FileSnap@82] - Reading snapshot /home/hadoop/zoo/dataDir/version-2/snapshot.0
2011-12-08 20:13:56,708 - INFO  [main:FileTxnSnapLog@208] - Snapshotting: 4e
2011-12-08 20:14:52,147 - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:40694
2011-12-08 20:14:52,153 - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@777] - Client attempting to establish new session at /0:0:0:0:0:0:0:1:40694
2011-12-08 20:14:52,154 - INFO  [SyncThread:0:FileTxnLog@197] - Creating new log file: log.4f
2011-12-08 20:14:52,410 - INFO  [SyncThread:0:NIOServerCnxn@1580] - Established session 0x13420623ee70000 with negotiated timeout 30000 for client /0:0:0:0:0:0:0:1:4069
4
2011-12-08 20:14:52,959 - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:38446
2011-12-08 20:14:52,962 - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@777] - Client attempting to establish new session at /127.0.0.1:38446
2011-12-08 20:14:53,007 - INFO  [SyncThread:0:NIOServerCnxn@1580] - Established session 0x13420623ee70001 with negotiated timeout 30000 for client /127.0.0.1:38446
2011-12-08 20:14:59,932 - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@634] - EndOfStreamException: Unable to read additional data from client session
id 0x13420623ee70000, likely client has closed socket

当我启动accumulo shell时,我收到以下错误:

18 12:44:38,746 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:38,846 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:38,947 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:39,048 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:39,148 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:39,249 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:39,350 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:39,450 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []

4 个答案:

答案 0 :(得分:2)

JVM允许将服务器内存设置更正为不超过waht。 Tserver不会崩溃并且错误已解决。 答案来自accumulo-incubator用户列表,并在底部重新发布。 基本上,当进入并修改内存设置以在笔记本电脑上以伪分布式模式运行时,我对accumulo-site.xml和accumulo-env.sh文件进行了有关平板电脑服务器内存使用情况的错误修改。可以在/home/hadoop/accumulo/logs/tserver*.log文件中找到错误的线索:

20 18:20:00,951 [tabletserver.NativeMap] ERROR: Failed to load native
map library
/home/hadoop/accumulo/lib/native/map/libNativeMap-Linux-i386-32.so
java.lang.UnsatisfiedLinkError: Can't load library:
/home/hadoop/accumulo/lib/native/map/libNativeMap-Linux-i386-32.so
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1706)
at java.lang.Runtime.load0(Runtime.java:770)
at java.lang.System.load(System.java:1003)
at
org.apache.accumulo.server.tabletserver.NativeMap.loadNativeLib(NativeMap.java:144)
at
org.apache.accumulo.server.tabletserver.NativeMap.<clinit>(NativeMap.java:156)
at
org.apache.accumulo.server.tabletserver.TabletServerResourceManager.<init>(TabletServerResourceManager.java:123)
at
org.apache.accumulo.server.tabletserver.TabletServer.config(TabletServer.java:2959)
at
org.apache.accumulo.server.tabletserver.TabletServer.main(TabletServer.java:3085)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.accumulo.start.Main$1.run(Main.java:89)
at java.lang.Thread.run(Thread.java:662)
20 18:20:00,999 [tabletserver.TabletServer] ERROR: Uncaught exception in
TabletServer.main, exiting
java.lang.IllegalArgumentException: Maximum tablet server map memory
134,217,728 and block cache sizes 186,646,528 is too large for this JVM
configuration 132,579,328
at
org.apache.accumulo.server.tabletserver.TabletServerResourceManager.<init>(TabletServerResourceManager.java:134)
at
org.apache.accumulo.server.tabletserver.TabletServer.config(TabletServer.java:2959)
at
org.apache.accumulo.server.tabletserver.TabletServer.main(TabletServer.java:3085)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.accumulo.start.Main$1.run(Main.java:89)
at java.lang.Thread.run(Thread.java:662)

具体帮助文字: “本机库没有加载,这是在内存中转移 映射到java工作区。添加你的块缓存大小和你的 内存使用规范高于JVM允许的规范 分配。平板电脑服务器抱怨并退出。你应该看到这些 关于累积显示器网页的投诉。

您可能会发现重建本机地图库的好处 将内存映射的分配移动到JVM外部。这是 不需要。

专用于缓存的任何内存的大小必须小于大小 JVM,必须包含大量的RPC调用工作空间 和垃圾收集随着时间的推移。“

答案 1 :(得分:0)

如果您运行Zookeeper客户端(/home/hadoop/zookeeper-3.3.3/bin/zkCli.sh)并执行 ls

/accumulo/<instance uuid>/tservers

我假设您不会看到列出的任何服务器。如果正确初始化Accumulo,您应该看到列出的一个或多个平板电脑服务器。根据说明在 accumulo-site.xml 中设置Zookeeper服务器后,您确定运行了Accumulo init script吗?

答案 2 :(得分:0)

确保将“instance.zookeeper.host”设置为accumulo-site.xml文件中zookeeper节点的位置。

此外,请检查您的服务器和记录器的日志。如果您有其他配置问题,它们将无法生效,这将导致主机报告难以找到服务器。

答案 3 :(得分:0)

进入$ ACCUMULO_HOME / conf。结束文件master,slave和tracers包含一行读取“localhost”的行(我假设你正在做单个节点)