运行几秒钟后,Storm拓扑在本地群集中关闭

时间:2019-12-24 02:08:59

标签: apache-storm

我有一个非常基本的拓扑。从KafkaSpout开始,它有3个螺栓。第一个螺栓是用于在Cassandra中写入数据的CassandraWriterBolt,其余两个螺栓从Cassandra读取旧数据,使用新旧数据创建另一组数据,然后再次将该数据插入Cassandra。

在开发过程中,我正在LocalCluster中运行该拓扑。它运行了几秒钟,然后开始关闭worker,executor等。最后,它失败了,并出现了Cassandra驱动程序相关的异常-

java.lang.IllegalStateException:无法发送请求,会话已关闭     在com.datastax.driver.core.SessionManager.execute(SessionManager.java:696)〜[cassandra-driver-core-3.6.0.jar:na]

其他日志是-

[er Executor - 1] o.a.s.s.org.apache.zookeeper.ZooKeeper   : Session: 0x100166ad36d0024 closed
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.server.NIOServerCnxn       : Unable to read additional data from client sessionid 0x100166ad36d0024, likely client has closed socket
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.server.NIOServerCnxn       : Closed socket connection for client /0:0:0:0:0:0:0:1:63890 which had sessionid 0x100166ad36d0024
[- 1-EventThread] o.a.s.s.org.apache.zookeeper.ClientCnxn  : EventThread shut down for session: 0x100166ad36d0024
[tor-Framework-0] o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl   : backgroundOperationsLoop exiting
[:0 cport:2000):] o.a.s.s.o.a.z.s.PrepRequestProcessor     : Processed session termination for sessionid: 0x100166ad36d0021
[er Executor - 4] o.a.s.s.org.apache.zookeeper.ZooKeeper   : Session: 0x100166ad36d0021 closed
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.server.NIOServerCnxn       : Unable to read additional data from client sessionid 0x100166ad36d0021, likely client has closed socket
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.server.NIOServerCnxn       : Closed socket connection for client /0:0:0:0:0:0:0:1:63885 which had sessionid 0x100166ad36d0021
[- 4-EventThread] o.a.s.s.org.apache.zookeeper.ClientCnxn  : EventThread shut down for session: 0x100166ad36d0021
[      SLOT_1027] org.apache.storm.ProcessSimulator        : Begin killing process 1347f01d-7982-4141-9b9d-cac65a6e703d
[      SLOT_1027] org.apache.storm.daemon.worker.Worker    : Shutting down worker forex-topology-1-1577152204 517f3306-5ad3-433b-82e1-b2d031779f0b 1027
[      SLOT_1027] org.apache.storm.daemon.worker.Worker    : Terminating messaging context
[      SLOT_1027] org.apache.storm.daemon.worker.Worker    : Shutting down executors
[      SLOT_1027] o.a.storm.executor.ExecutorShutdown      : Shutting down executor __system:[-1, -1]
[xecutor[-1, -1]] org.apache.storm.utils.Utils             : Async loop interrupted!
[      SLOT_1027] o.a.storm.executor.ExecutorShutdown      : Shut down executor __system:[-1, -1]
[      SLOT_1027] o.a.storm.executor.ExecutorShutdown      : Shutting down executor pairStrengthAccumulator:[8, 8]
[-executor[8, 8]] org.apache.storm.utils.Utils             : Async loop interrupted!
[      SLOT_1027] o.a.s.cassandra.executor.AsyncExecutor   : shutting down async handler executor
[      SLOT_1027] o.a.s.c.client.impl.DefaultClient        : Try to close connection to cluster: cluster2

以下日志可以查看40次-

[           main] o.a.storm.zookeeper.ClientZookeeper      : Starting ZK Curator
[           main] o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl   : Starting
[           main] o.a.s.s.org.apache.zookeeper.ZooKeeper   : Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@4bcaa195
[           main] o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl   : Default schema
[localhost:2000)] o.a.s.s.org.apache.zookeeper.ClientCnxn  : Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2000. Will not attempt to authenticate using SASL (unknown error)
[           main] o.a.storm.zookeeper.ClientZookeeper      : Starting ZK Curator
[           main] o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl   : Starting
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.s.NIOServerCnxnFactory     : Accepted socket connection from /0:0:0:0:0:0:0:1:63756
[           main] o.a.s.s.org.apache.zookeeper.ZooKeeper   : Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@6bc24e72
[localhost:2000)] o.a.s.s.org.apache.zookeeper.ClientCnxn  : Socket connection established to localhost/0:0:0:0:0:0:0:1:2000, initiating session
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.server.ZooKeeperServer     : Client attempting to establish new session at /0:0:0:0:0:0:0:1:63756
[           main] o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl   : Default schema
[localhost:2000)] o.a.s.s.org.apache.zookeeper.ClientCnxn  : Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2000, sessionid = 0x100166ad36d0001, negotiated timeout = 20000
[   SyncThread:0] o.a.s.s.o.a.z.server.ZooKeeperServer     : Established session 0x100166ad36d0001 with negotiated timeout 20000 for client /0:0:0:0:0:0:0:1:63756
[localhost:2000)] o.a.s.s.org.apache.zookeeper.ClientCnxn  : Opening socket connection to server localhost/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error)
[ain-EventThread] o.a.s.s.o.a.c.f.s.ConnectionStateManager : State change: CONNECTED
[localhost:2000)] o.a.s.s.org.apache.zookeeper.ClientCnxn  : Socket connection established to localhost/127.0.0.1:2000, initiating session
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.s.NIOServerCnxnFactory     : Accepted socket connection from /127.0.0.1:63759
[.0/0.0.0.0:2000] o.a.s.s.o.a.z.server.ZooKeeperServer     : Client attempting to establish new session at /127.0.0.1:63759
[   SyncThread:0] o.a.s.s.o.a.z.server.ZooKeeperServer     : Established session 0x100166ad36d0002 with negotiated timeout 20000 for client /127.0.0.1:63759
[localhost:2000)] o.a.s.s.org.apache.zookeeper.ClientCnxn  : Session establishment complete on server localhost/127.0.0.1:2000, sessionid = 0x100166ad36d0002, negotiated timeout = 20000
[ain-EventThread] o.a.s.s.o.a.c.f.s.ConnectionStateManager : State change: CONNECTED
[           main] o.a.storm.validation.ConfigValidation    : task.heartbeat.frequency.secs is a deprecated config please see class org.apache.storm.Config.TASK_HEARTBEAT_FREQUENCY_SECS for more information.

1 个答案:

答案 0 :(得分:0)

您的主要方法是这样做的:

public static void main(String[] args) {
        ApplicationContext springContext = SpringApplication.run(CurrencyStrengthCalculatorApplication.class, args);
        StormTopology topology = SpringBasedTopologyBuilder.getInstance().buildStormTopologyUsingApplicationContext(springContext);
        LOG.info("Topology created successfully. Now starting it .............");
        new LocalCluster().submitTopology("forext-topology", ImmutableMap.of(), topology);
    }

submitTopology不是阻塞调用,它只是提交拓扑并返回。如果要使程序运行一段时间,则需要在提交后进入睡眠状态。一旦main方法返回,LocalCluster将开始关闭。