SchedulerStateManagerAdaptor无法从苍鹭群集中的Zookeeper路径获取数据
在Heron中，我使用SchedulerStateManagerAdaptor实例化了一个名为adaptor的{{1}}，并创建了一个新的IStateManager配置来保存该适配器。然后，我创建了一个Runtime对象来调用其ISchedulerClient函数。代码如下：
schedulerClient.updateTopology但是，当我尝试在实例化调度程序调用的String stateMgrClass = Context.stateManagerClass(this.config); // get state manager instance
IStateManager stateMgr = null;
try {
stateMgr = ReflectionUtils.newInstance(stateMgrClass);
FileUtils.writeToFile(filename, "Create IStateManager object success...");
} catch (ClassNotFoundException | InstantiationException | IllegalAccessException e) {
e.printStackTrace();
}
stateMgr.initialize(this.config);
stateManagerAdaptor = new SchedulerStateManagerAdaptor(stateMgr, 5000);
Config runtime = Config.newBuilder()
.putAll(primaryRuntime)
.put(Key.TOPOLOGY_NAME, Context.topologyName(config))
.put(Key.SCHEDULER_STATE_MANAGER_ADAPTOR, stateManagerAdaptor)
.build();
if (!schedulerClient.updateTopology(updateTopologyRequest)) {
throw new TopologyRuntimeManagementException(String.format(
"Failed to update " + topology.getName() +" with Scheduler, updateTopologyRequest=" + updateTopologyRequest));
}
函数中进行adaptor.getPhysicalPlan时。发生此异常：
而且，我在zookeeper日志中发现了异常：
scheduler.onUpdate我不知道这是怎么回事！但是，当我在创建2018-07-07 21:01:09,855 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /218.195.228.24:41348 which had sessionid 0x16469a49d550f23
2018-07-07 21:01:12,001 [myid:] - INFO [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f29, timeout of 4000ms exceeded
2018-07-07 21:01:12,002 [myid:] - INFO [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f28, timeout of 4000ms exceeded
2018-07-07 21:01:12,002 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f29
2018-07-07 21:01:12,003 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f28
2018-07-07 21:01:20,001 [myid:] - INFO [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f2a, timeout of 10000ms exceeded
2018-07-07 21:01:20,003 [myid:] - INFO [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f24, timeout of 10000ms exceeded
2018-07-07 21:01:20,003 [myid:] - INFO [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f23, timeout of 10000ms exceeded
2018-07-07 21:01:20,003 [myid:] - INFO [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16474d4e4bb0003, timeout of 10000ms exceeded
2018-07-07 21:01:20,004 [myid:] - INFO [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16474d4e4bb0004, timeout of 10000ms exceeded
2018-07-07 21:01:20,005 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f2a
2018-07-07 21:01:20,005 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f24
2018-07-07 21:01:20,006 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f23
2018-07-07 21:01:20,006 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16474d4e4bb0003
2018-07-07 21:01:20,006 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16474d4e4bb0004
2018-07-07 21:01:23,779 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41378
2018-07-07 21:01:23,779 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /218.195.228.24:41378; will be dropped if server is in r-o mode
2018-07-07 21:01:23,779 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41378
2018-07-07 21:01:23,780 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41380
2018-07-07 21:01:23,780 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /218.195.228.24:41380; will be dropped if server is in r-o mode
2018-07-07 21:01:23,780 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41380
2018-07-07 21:01:23,783 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41382
2018-07-07 21:01:23,783 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /218.195.228.24:41382; will be dropped if server is in r-o mode
2018-07-07 21:01:23,783 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41382
2018-07-07 21:01:23,784 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0005 with negotiated timeout 10000 for client /218.195.228.24:41378
2018-07-07 21:01:23,785 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41384
2018-07-07 21:01:23,785 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /218.195.228.24:41384; will be dropped if server is in r-o mode
2018-07-07 21:01:23,785 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41384
2018-07-07 21:01:23,786 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0006 with negotiated timeout 10000 for client /218.195.228.24:41380
2018-07-07 21:01:23,787 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0007 with negotiated timeout 10000 for client /218.195.228.24:41382
2018-07-07 21:01:23,789 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0008 with negotiated timeout 10000 for client /218.195.228.24:41384
2018-07-07 21:01:38,005 [myid:] - INFO [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f70, timeout of 30000ms exceeded
2018-07-07 21:01:38,005 [myid:] - INFO [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f71, timeout of 30000ms exceeded
2018-07-07 21:01:38,005 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f70
2018-07-07 21:01:38,005 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f71
2018-07-07 21:02:19,897 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41428
2018-07-07 21:02:19,899 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41428
2018-07-07 21:02:19,900 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0009 with negotiated timeout 30000 for client /218.195.228.24:41428
2018-07-07 21:02:26,397 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41436
2018-07-07 21:02:26,398 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41436
2018-07-07 21:02:26,399 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb000a with negotiated timeout 10000 for client /218.195.228.24:41436
2018-07-07 21:02:26,403 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16474d4e4bb000a
2018-07-07 21:02:26,404 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /218.195.228.24:41436 which had sessionid 0x16474d4e4bb000a
2018-07-07 21:02:29,672 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.43:40088
2018-07-07 21:02:29,675 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.43:40088
2018-07-07 21:02:29,679 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb000b with negotiated timeout 10000 for client /218.195.228.43:40088
2018-07-07 21:02:30,480 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.28:57174
2018-07-07 21:02:30,493 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.28:57174
2018-07-07 21:02:30,498 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb000c with negotiated timeout 10000 for client /218.195.228.28:57174
2018-07-07 21:02:31,233 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.19:52872
2018-07-07 21:02:31,235 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.19:52872
2018-07-07 21:02:31,239 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb000d with negotiated timeout 10000 for client /218.195.228.19:52872
2018-07-07 21:02:34,002 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16474d4e4bb0009
2018-07-07 21:02:34,009 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /218.195.228.24:41428 which had sessionid 0x16474d4e4bb0009
2018-07-07 21:02:57,272 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.43:40100
2018-07-07 21:02:57,274 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@368] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
at java.lang.Thread.run(Thread.java:748)
2018-07-07 21:02:57,275 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /218.195.228.43:40100 (no session established for client)
2018-07-07 21:02:57,276 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.43:40102
2018-07-07 21:02:57,277 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.43:40102
2018-07-07 21:02:57,280 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb000e with negotiated timeout 10000 for client /218.195.228.43:40102
2018-07-07 21:02:57,357 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.43:40106
2018-07-07 21:02:57,358 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /218.195.228.43:40106; will be dropped if server is in r-o mode
2018-07-07 21:02:57,358 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.43:40106
2018-07-07 21:02:57,362 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb000f with negotiated timeout 30000 for client /218.195.228.43:40106
2018-07-07 21:02:58,936 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.43:40110
2018-07-07 21:02:58,938 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.43:40110
2018-07-07 21:02:58,943 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0010 with negotiated timeout 30000 for client /218.195.228.43:40110
2018-07-07 21:03:05,562 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.19:52884
2018-07-07 21:03:05,564 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@368] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
at java.lang.Thread.run(Thread.java:748)
2018-07-07 21:03:05,565 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /218.195.228.19:52884 (no session established for client)
2018-07-07 21:03:05,568 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.19:52886
2018-07-07 21:03:05,569 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.19:52886
2018-07-07 21:03:05,575 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0011 with negotiated timeout 10000 for client /218.195.228.19:52886
2018-07-07 21:03:05,794 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.19:52888
2018-07-07 21:03:05,795 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /218.195.228.19:52888; will be dropped if server is in r-o mode
2018-07-07 21:03:05,796 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.19:52888
2018-07-07 21:03:05,798 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0012 with negotiated timeout 30000 for client /218.195.228.19:52888
2018-07-07 21:03:43,958 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41544
2018-07-07 21:03:43,960 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41544
2018-07-07 21:03:43,984 [myid:] - INFO [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0013 with negotiated timeout 30000 for client /218.195.228.24:41544
2018-07-07 21:03:44,041 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16474d4e4bb0013
2018-07-07 21:03:44,043 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /218.195.228.24:41544 which had sessionid 0x16474d4e4bb0013
2018-07-07 21:03:53,413 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.28:57186
2018-07-07 21:03:53,415 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@368] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
at java.lang.Thread.run(Thread.java:748)
对象的文件中调用adaptor.getPhysicalPlan时，它就起作用了。是什么原因导致此问题？感谢您的帮助。
0 个答案:
- 跨2 zookeeper集群复制数据的最佳方法？
- 如何将Kafka从旧的Zookeeper集群迁移到具有不同znode父路径的新Zookeeper集群
- 尝试从zookeeper节点获取数据时出现异常：（KeeperErrorCode：ConnectionLoss）
- 苍鹭无法为拓扑'WordCountTopology'设置包装计划
- 使用AuroraScheduler在Heron群集中不输出日志
- 在Heron群集中更新拓扑失败
- SchedulerStateManagerAdaptor无法从苍鹭群集中的Zookeeper路径获取数据
- Zookeeper和Heron EndOfStreamException：无法从客户端读取其他数据
- Zookeeper EndOfStreamException发生在苍鹭群集中
- 如何监控苍鹭群集的吞吐量
- 我写了这段代码，但我无法理解我的错误
- 我无法从一个代码实例的列表中删除 None 值，但我可以在另一个实例中。为什么它适用于一个细分市场而不适用于另一个细分市场？
- 是否有可能使 loadstring 不可能等于打印？卢阿
- java中的random.expovariate()
- Appscript 通过会议在 Google 日历中发送电子邮件和创建活动
- 为什么我的 Onclick 箭头功能在 React 中不起作用？
- 在此代码中是否有使用“this”的替代方法？
- 在 SQL Server 和 PostgreSQL 上查询，我如何从第一个表获得第二个表的可视化
- 每千个数字得到
- 更新了城市边界 KML 文件的来源？