我有一个非常基本的拓扑。从KafkaSpout开始,它有3个螺栓。第一个螺栓是用于在Cassandra中写入数据的CassandraWriterBolt,其余两个螺栓从Cassandra读取旧数据,使用新旧数据创建另一组数据,然后再次将该数据插入Cassandra。
在开发过程中,我正在LocalCluster中运行该拓扑。它运行了几秒钟,然后开始关闭worker,executor等。最后,它失败了,并出现了Cassandra驱动程序相关的异常-
java.lang.IllegalStateException:无法发送请求,会话已关闭 在com.datastax.driver.core.SessionManager.execute(SessionManager.java:696)〜[cassandra-driver-core-3.6.0.jar:na]
其他日志是-
[er Executor - 1] o.a.s.s.org.apache.zookeeper.ZooKeeper : Session: 0x100166ad36d0024 closed
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.server.NIOServerCnxn : Unable to read additional data from client sessionid 0x100166ad36d0024, likely client has closed socket
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.server.NIOServerCnxn : Closed socket connection for client /0:0:0:0:0:0:0:1:63890 which had sessionid 0x100166ad36d0024
[- 1-EventThread] o.a.s.s.org.apache.zookeeper.ClientCnxn : EventThread shut down for session: 0x100166ad36d0024
[tor-Framework-0] o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl : backgroundOperationsLoop exiting
[:0 cport:2000):] o.a.s.s.o.a.z.s.PrepRequestProcessor : Processed session termination for sessionid: 0x100166ad36d0021
[er Executor - 4] o.a.s.s.org.apache.zookeeper.ZooKeeper : Session: 0x100166ad36d0021 closed
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.server.NIOServerCnxn : Unable to read additional data from client sessionid 0x100166ad36d0021, likely client has closed socket
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.server.NIOServerCnxn : Closed socket connection for client /0:0:0:0:0:0:0:1:63885 which had sessionid 0x100166ad36d0021
[- 4-EventThread] o.a.s.s.org.apache.zookeeper.ClientCnxn : EventThread shut down for session: 0x100166ad36d0021
[ SLOT_1027] org.apache.storm.ProcessSimulator : Begin killing process 1347f01d-7982-4141-9b9d-cac65a6e703d
[ SLOT_1027] org.apache.storm.daemon.worker.Worker : Shutting down worker forex-topology-1-1577152204 517f3306-5ad3-433b-82e1-b2d031779f0b 1027
[ SLOT_1027] org.apache.storm.daemon.worker.Worker : Terminating messaging context
[ SLOT_1027] org.apache.storm.daemon.worker.Worker : Shutting down executors
[ SLOT_1027] o.a.storm.executor.ExecutorShutdown : Shutting down executor __system:[-1, -1]
[xecutor[-1, -1]] org.apache.storm.utils.Utils : Async loop interrupted!
[ SLOT_1027] o.a.storm.executor.ExecutorShutdown : Shut down executor __system:[-1, -1]
[ SLOT_1027] o.a.storm.executor.ExecutorShutdown : Shutting down executor pairStrengthAccumulator:[8, 8]
[-executor[8, 8]] org.apache.storm.utils.Utils : Async loop interrupted!
[ SLOT_1027] o.a.s.cassandra.executor.AsyncExecutor : shutting down async handler executor
[ SLOT_1027] o.a.s.c.client.impl.DefaultClient : Try to close connection to cluster: cluster2
以下日志可以查看40次-
[ main] o.a.storm.zookeeper.ClientZookeeper : Starting ZK Curator
[ main] o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl : Starting
[ main] o.a.s.s.org.apache.zookeeper.ZooKeeper : Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@4bcaa195
[ main] o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl : Default schema
[localhost:2000)] o.a.s.s.org.apache.zookeeper.ClientCnxn : Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2000. Will not attempt to authenticate using SASL (unknown error)
[ main] o.a.storm.zookeeper.ClientZookeeper : Starting ZK Curator
[ main] o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl : Starting
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.s.NIOServerCnxnFactory : Accepted socket connection from /0:0:0:0:0:0:0:1:63756
[ main] o.a.s.s.org.apache.zookeeper.ZooKeeper : Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@6bc24e72
[localhost:2000)] o.a.s.s.org.apache.zookeeper.ClientCnxn : Socket connection established to localhost/0:0:0:0:0:0:0:1:2000, initiating session
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.server.ZooKeeperServer : Client attempting to establish new session at /0:0:0:0:0:0:0:1:63756
[ main] o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl : Default schema
[localhost:2000)] o.a.s.s.org.apache.zookeeper.ClientCnxn : Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2000, sessionid = 0x100166ad36d0001, negotiated timeout = 20000
[ SyncThread:0] o.a.s.s.o.a.z.server.ZooKeeperServer : Established session 0x100166ad36d0001 with negotiated timeout 20000 for client /0:0:0:0:0:0:0:1:63756
[localhost:2000)] o.a.s.s.org.apache.zookeeper.ClientCnxn : Opening socket connection to server localhost/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error)
[ain-EventThread] o.a.s.s.o.a.c.f.s.ConnectionStateManager : State change: CONNECTED
[localhost:2000)] o.a.s.s.org.apache.zookeeper.ClientCnxn : Socket connection established to localhost/127.0.0.1:2000, initiating session
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.s.NIOServerCnxnFactory : Accepted socket connection from /127.0.0.1:63759
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.server.ZooKeeperServer : Client attempting to establish new session at /127.0.0.1:63759
[ SyncThread:0] o.a.s.s.o.a.z.server.ZooKeeperServer : Established session 0x100166ad36d0002 with negotiated timeout 20000 for client /127.0.0.1:63759
[localhost:2000)] o.a.s.s.org.apache.zookeeper.ClientCnxn : Session establishment complete on server localhost/127.0.0.1:2000, sessionid = 0x100166ad36d0002, negotiated timeout = 20000
[ain-EventThread] o.a.s.s.o.a.c.f.s.ConnectionStateManager : State change: CONNECTED
[ main] o.a.storm.validation.ConfigValidation : task.heartbeat.frequency.secs is a deprecated config please see class org.apache.storm.Config.TASK_HEARTBEAT_FREQUENCY_SECS for more information.
答案 0 :(得分:0)
您的主要方法是这样做的:
public static void main(String[] args) {
ApplicationContext springContext = SpringApplication.run(CurrencyStrengthCalculatorApplication.class, args);
StormTopology topology = SpringBasedTopologyBuilder.getInstance().buildStormTopologyUsingApplicationContext(springContext);
LOG.info("Topology created successfully. Now starting it .............");
new LocalCluster().submitTopology("forext-topology", ImmutableMap.of(), topology);
}
submitTopology
不是阻塞调用,它只是提交拓扑并返回。如果要使程序运行一段时间,则需要在提交后进入睡眠状态。一旦main方法返回,LocalCluster将开始关闭。