当我将WordCountTopology提交到使用Aurora调度程序和Zookeeper部署的Heron Cluster时,此错误发生如下:
yitian@heron01:~$ heron submit aurora/yitian/devel --config-path ~/.heron/conf ~/.heron/examples/heron-api-examples.jar com.twitter.heron.examples.api.WordCountTopology WordCountTopology --deploy-deactivated
[2018-06-04 00:55:54 +0000] [INFO]: Using cluster definition in /home/yitian/.heron/conf/aurora
[2018-06-04 00:55:54 +0000] [INFO]: Launching topology: 'WordCountTopology'
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/yitian/.heron/lib/uploader/heron-dlog-uploader.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/yitian/.heron/lib/statemgr/heron-zookeeper-statemgr.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
[2018-06-04 00:55:55 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting Curator client connecting to: heron01:2181
[2018-06-04 00:55:55 -0700] [INFO] org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:host.name=heron01
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.8.0_151
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/java/jdk1.8.0_151/jre
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=:/home/yitian/.heron/lib/scheduler/heron-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-binpacking-packing.jar:/home/yitian/.heron/lib/scheduler/heron-yarn-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-slurm-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-mesos-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-marathon-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-kubernetes-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-roundrobin-packing.jar:/home/yitian/.heron/lib/scheduler/heron-local-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-aurora-scheduler.jar:/home/yitian/.heron/lib/uploader/heron-localfs-uploader.jar:/home/yitian/.heron/lib/uploader/heron-hdfs-uploader.jar:/home/yitian/.heron/lib/uploader/heron-null-uploader.jar:/home/yitian/.heron/lib/uploader/heron-scp-uploader.jar:/home/yitian/.heron/lib/uploader/heron-s3-uploader.jar:/home/yitian/.heron/lib/uploader/heron-gcs-uploader.jar:/home/yitian/.heron/lib/uploader/heron-dlog-uploader.jar:/home/yitian/.heron/lib/statemgr/heron-localfs-statemgr.jar:/home/yitian/.heron/lib/statemgr/heron-zookeeper-statemgr.jar:/home/yitian/.heron/lib/packing/heron-binpacking-packing.jar:/home/yitian/.heron/lib/packing/heron-roundrobin-packing.jar
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:os.version=4.10.0-28-generic
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:user.name=yitian
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:user.home=/home/yitian
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/home/yitian
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=heron01:2181 sessionTimeout=30000 watcher=org.apache.curator.ConnectionState@2a17b7b6
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ClientCnxn: Opening socket connection to server heron01/218.195.241.174:2181. Will not attempt to authenticate using SASL (unknown error)
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ClientCnxn: Socket connection established to heron01/218.195.241.174:2181, initiating session
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ClientCnxn: Session establishment complete on server heron01/218.195.241.174:2181, sessionid = 0x163c9cb80580003, negotiated timeout = 30000
[2018-06-04 00:55:55 -0700] [INFO] org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED
[2018-06-04 00:55:55 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree initialized.
[2018-06-04 00:55:55 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Checking existence of path: /heron/topologies/WordCountTopology
[2018-06-04 00:55:58 -0700] [INFO] com.twitter.heron.uploader.hdfs.HdfsUploader: Target topology file already exists at '/heron/topologies/aurora/WordCountTopology-yitian-tag-0--1244338252177277003.tar.gz'. Overwriting it now
[2018-06-04 00:55:58 -0700] [INFO] com.twitter.heron.uploader.hdfs.HdfsUploader: Uploading topology package at '/tmp/tmpXlYukj/topology.tar.gz' to target HDFS at '/heron/topologies/aurora/WordCountTopology-yitian-tag-0--1244338252177277003.tar.gz'
[2018-06-04 00:56:02 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node for path: /heron/topologies/WordCountTopology
[2018-06-04 00:56:02 -0700] [WARNING] com.twitter.heron.spi.statemgr.SchedulerStateManagerAdaptor: Exception processing future: java.lang.RuntimeException: Could not createNode:
[2018-06-04 00:56:02 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted node for path: /heron/topologies/WordCountTopology
18/06/04 00:56:05 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.
Deleted /heron/topologies/aurora/WordCountTopology-yitian-tag-0--1244338252177277003.tar.gz
[2018-06-04 00:56:05 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the CuratorClient to: heron01:2181
[2018-06-04 00:56:05 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Session: 0x163c9cb80580003 closed
[2018-06-04 00:56:05 -0700] [INFO] org.apache.zookeeper.ClientCnxn: EventThread shut down
[2018-06-04 00:56:05 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the tunnel processes
[2018-06-04 00:56:05 +0000] [ERROR]: Failed to set packing plan for topology 'WordCountTopology'
[2018-06-04 00:56:05 +0000] [ERROR]: Failed to launch topology 'WordCountTopology'
它出了什么问题?谢谢你的帮助。
答案 0 :(得分:2)
来自日志:
[2018-06-04 00:56:02 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager:路径的创建节点:/ heron / topologies / WordCountTopology
[2018-06-04 00:56:02 -0700] [警告] com.twitter.heron.spi.statemgr.SchedulerStateManagerAdaptor:异常处理未来:java.lang.RuntimeException:无法创建节点:
Heron能够连接到ZK并成功检查节点,因此ZK集群似乎正在运行且可读。然而,由于某种原因,策展人无法在ZK中创建节点:/ heron /拓扑/ WordCountTopology
Heron拓扑将其运行数据保存在ZK中,因此如果无法创建关键节点,则无法启动。你需要找出ZK失败的原因(权限?)并解决它。