在StormSubmitter.submitJar（...）用于提交Apache Storm拓扑时重置连接

时间：2018-06-26 05:30:30

标签： apache-zookeeper apache-storm apache-storm-topology

无法使用StormSubmitter.submitJar（...）

提交JAR
  • 开始上传，但一段时间后出现错误。 LocalCluster设置
  • 可以正常工作，只有在群集的情况下才能使用 泊坞窗容器。
  • 瓶子的大小约为132MB。
  • 尝试使用Apache Storm的1.2.1和1.2.2版本。
  • 尝试使用1个ZK，1个Nimbus，1个Supervisor和1个Storm UI作为docker容器节点。
  • Nimbus日志中没有错误。
  • 尝试了 nimbus.thrift.max_buffer_size ，即大量使用，即908914889

获得以下异常：

1894 [main] INFO  c.a.m.b.m.StormProgressListener - UPLOADPROGRESS: localJar: /./storm-topology-0.0.1.jar ,uploadLocation: /storm/localdir/nimbus/inbox/stormjar-dd48f935-58c7-44a8-916e-8f1eafc4fe8d.jar ,bytesUploaded: 48230400 ,totalSize: 108517419
1902 [main] INFO  c.a.m.b.m.StormProgressListener - UPLOADPROGRESS: localJar: /./storm-topology-0.0.1.jar ,uploadLocation: /storm/localdir/nimbus/inbox/stormjar-dd48f935-58c7-44a8-916e-8f1eafc4fe8d.jar ,bytesUploaded: 48537600 ,totalSize: 108517419
1921 [main] INFO  c.a.m.b.m.StormProgressListener - UPLOADPROGRESS: localJar: /./storm-topology-0.0.1.jar ,uploadLocation: /storm/localdir/nimbus/inbox/stormjar-dd48f935-58c7-44a8-916e-8f1eafc4fe8d.jar ,bytesUploaded: 48844800 ,totalSize: 108517419
2026 [main] ERROR c.a.m.b.m.TopologySubmitter - Error running topology: 
java.lang.RuntimeException: org.apache.storm.thrift.transport.TTransportException: java.net.SocketException: Connection reset
    at org.apache.storm.StormSubmitter.submitJarAs(StormSubmitter.java:503) ~[app.jar:?]
    at org.apache.storm.StormSubmitter.submitJarAs(StormSubmitter.java:513) ~[app.jar:?]
    at org.apache.storm.StormSubmitter.submitJar(StormSubmitter.java:525) ~[app.jar:?]
    at a.b.c.d.main.TopologySubmitter.submitTopologies(TopologySubmitter.java:68) [app.jar:?]
    at a.b.c.d.main.TopologySubmitter.main(TopologySubmitter.java:40) [app.jar:?]
Caused by: org.apache.storm.thrift.transport.TTransportException: java.net.SocketException: Connection reset
    at org.apache.storm.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129) ~[app.jar:?]
    at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86) ~[app.jar:?]
    at org.apache.storm.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129) ~[app.jar:?]
    at org.apache.storm.thrift.transport.TFramedTransport.read(TFramedTransport.java:101) ~[app.jar:?]
    at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86) ~[app.jar:?]
    at org.apache.storm.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) ~[app.jar:?]
    at org.apache.storm.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) ~[app.jar:?]
    at org.apache.storm.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) ~[app.jar:?]
    at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:77) ~[app.jar:?]
    at org.apache.storm.generated.Nimbus$Client.recv_uploadChunk(Nimbus.java:1046) ~[app.jar:?]
    at org.apache.storm.generated.Nimbus$Client.uploadChunk(Nimbus.java:1032) ~[app.jar:?]
    at org.apache.storm.StormSubmitter.submitJarAs(StormSubmitter.java:492) ~[app.jar:?]
    ... 4 more
Caused by: java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:210) ~[?:1.8.0_171]
    at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_171]
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[?:1.8.0_171]
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[?:1.8.0_171]
    at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[?:1.8.0_171]
    at org.apache.storm.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) ~[app.jar:?]
    at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86) ~[app.jar:?]
    at org.apache.storm.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129) ~[app.jar:?]
    at org.apache.storm.thrift.transport.TFramedTransport.read(TFramedTransport.java:101) ~[app.jar:?]
    at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86) ~[app.jar:?]
    at org.apache.storm.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) ~[app.jar:?]
    at org.apache.storm.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) ~[app.jar:?]
    at org.apache.storm.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) ~[app.jar:?]
    at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:77) ~[app.jar:?]
    at org.apache.storm.generated.Nimbus$Client.recv_uploadChunk(Nimbus.java:1046) ~[app.jar:?]
    at org.apache.storm.generated.Nimbus$Client.uploadChunk(Nimbus.java:1032) ~[app.jar:?]
    at org.apache.storm.StormSubmitter.submitJarAs(StormSubmitter.java:492) ~[app.jar:?]
    ... 4 more

每次我尝试提交JAR（）时，我都在 zookeeper 中发现了奇怪的日志，这可能会有所帮助：

2018-06-26T06:59:47.72131895Z 32438 Health is good
2018-06-26T06:59:57.720821225Z 32439 Health is good
2018-06-26T07:00:05.068319125Z mx-storm-zookeeper1 20 [2018-06-26 07:00:05,068] WARN caught end of stream exception (org.apache.zookeeper.server.NIOServerCnxn)
2018-06-26T07:00:05.068350952Z mx-storm-zookeeper1 20 EndOfStreamException: Unable to read additional data from client sessionid 0x16427b3322603bc, likely client has closed s
ocket
2018-06-26T07:00:05.068361585Z mx-storm-zookeeper1 20   at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
2018-06-26T07:00:05.068370636Z mx-storm-zookeeper1 20   at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
2018-06-26T07:00:05.068378713Z mx-storm-zookeeper1 20   at java.lang.Thread.run(Thread.java:745)
2018-06-26T07:00:05.068636068Z mx-storm-zookeeper1 20 [2018-06-26 07:00:05,068] INFO Closed socket connection for client /10.10.0.153:60762 which had sessionid 0x16427b332260
3bc (org.apache.zookeeper.server.NIOServerCnxn)
2018-06-26T07:00:05.06876096Z mx-storm-zookeeper1 20 [2018-06-26 07:00:05,068] WARN caught end of stream exception (org.apache.zookeeper.server.NIOServerCnxn)
2018-06-26T07:00:05.068781679Z mx-storm-zookeeper1 20 EndOfStreamException: Unable to read additional data from client sessionid 0x16427b3322603bd, likely client has closed s
ocket
2018-06-26T07:00:05.068790511Z mx-storm-zookeeper1 20   at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
2018-06-26T07:00:05.068813111Z mx-storm-zookeeper1 20   at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
2018-06-26T07:00:05.068826053Z mx-storm-zookeeper1 20   at java.lang.Thread.run(Thread.java:745)
2018-06-26T07:00:05.069083608Z mx-storm-zookeeper1 20 [2018-06-26 07:00:05,069] INFO Closed socket connection for client /10.10.0.153:60764 which had sessionid 0x16427b3322603bd (org.apache.zookeeper.server.NIOServerCnxn)
2018-06-26T07:00:05.069219575Z mx-storm-zookeeper1 20 [2018-06-26 07:00:05,069] WARN caught end of stream exception (org.apache.zookeeper.server.NIOServerCnxn)
2018-06-26T07:00:05.069235697Z mx-storm-zookeeper1 20 EndOfStreamException: Unable to read additional data from client sessionid 0x16427b3322603ba, likely client has closed socket
2018-06-26T07:00:05.069244087Z mx-storm-zookeeper1 20   at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
2018-06-26T07:00:05.069254455Z mx-storm-zookeeper1 20   at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
2018-06-26T07:00:05.069264657Z mx-storm-zookeeper1 20   at java.lang.Thread.run(Thread.java:745)
2018-06-26T07:00:05.069275383Z mx-storm-zookeeper1 20 [2018-06-26 07:00:05,069] INFO Closed socket connection for client /10.10.0.153:60760 which had sessionid 0x16427b3322603ba (org.apache.zookeeper.server.NIOServerCnxn)
2018-06-26T07:00:05.069455649Z mx-storm-zookeeper1 20 [2018-06-26 07:00:05,069] WARN caught end of stream exception (org.apache.zookeeper.server.NIOServerCnxn)
2018-06-26T07:00:05.06947159Z mx-storm-zookeeper1 20 EndOfStreamException: Unable to read additional data from client sessionid 0x16427b3322603bb, likely client has closed socket
2018-06-26T07:00:05.069504647Z mx-storm-zookeeper1 20   at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
2018-06-26T07:00:05.069516266Z mx-storm-zookeeper1 20   at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
2018-06-26T07:00:05.069533746Z mx-storm-zookeeper1 20   at java.lang.Thread.run(Thread.java:745)
2018-06-26T07:00:05.069544746Z mx-storm-zookeeper1 20 [2018-06-26 07:00:05,069] INFO Closed socket connection for client /10.10.0.153:60756 which had sessionid 0x16427b3322603bb (org.apache.zookeeper.server.NIOServerCnxn)
2018-06-26T07:00:07.720847073Z 32440 Health is good
2018-06-26T07:00:17.720725011Z 32441 Health is good
2018-06-26T07:00:27.721569616Z 32442 Health is good
2018-06-26T07:00:34.001019673Z mx-storm-zookeeper1 20 [2018-06-26 07:00:34,000] INFO Expiring session 0x16427b3322603bb, timeout of 40000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:34.001056576Z mx-storm-zookeeper1 20 [2018-06-26 07:00:34,000] INFO Processed session termination for sessionid: 0x16427b3322603bb (org.apache.zookeeper.server.PrepRequestProcessor)
2018-06-26T07:00:36.001211361Z mx-storm-zookeeper1 20 [2018-06-26 07:00:36,001] INFO Expiring session 0x16427b3322603bc, timeout of 40000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:36.001400184Z mx-storm-zookeeper1 20 [2018-06-26 07:00:36,001] INFO Processed session termination for sessionid: 0x16427b3322603bc (org.apache.zookeeper.server.PrepRequestProcessor)
2018-06-26T07:00:37.720778634Z 32443 Health is good
2018-06-26T07:00:46.000929256Z mx-storm-zookeeper1 20 [2018-06-26 07:00:46,000] INFO Expiring session 0x16427b3322603bd, timeout of 40000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:46.000958587Z mx-storm-zookeeper1 20 [2018-06-26 07:00:46,000] INFO Expiring session 0x16427b3322603ba, timeout of 40000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:46.000973109Z mx-storm-zookeeper1 20 [2018-06-26 07:00:46,000] INFO Processed session termination for sessionid: 0x16427b3322603bd (org.apache.zookeeper.server.PrepRequestProcessor)
2018-06-26T07:00:46.001155085Z mx-storm-zookeeper1 20 [2018-06-26 07:00:46,001] INFO Processed session termination for sessionid: 0x16427b3322603ba (org.apache.zookeeper.server.PrepRequestProcessor)
2018-06-26T07:00:47.720717071Z 32444 Health is good
2018-06-26T07:00:48.471345397Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,471] INFO Accepted socket connection from /10.10.1.110:48850 (org.apache.zookeeper.server.NIOServerCnxnFactory)
2018-06-26T07:00:48.471382686Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,471] INFO Accepted socket connection from /10.10.1.110:48847 (org.apache.zookeeper.server.NIOServerCnxnFactory)
2018-06-26T07:00:48.471432303Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,471] INFO Accepted socket connection from /10.10.1.110:48846 (org.apache.zookeeper.server.NIOServerCnxnFactory)
2018-06-26T07:00:48.476131719Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,476] INFO Client attempting to establish new session at /10.10.1.110:48847 (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:48.4761533Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,476] INFO Client attempting to establish new session at /10.10.1.110:48850 (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:48.476167769Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,476] INFO Client attempting to establish new session at /10.10.1.110:48846 (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:48.477337666Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,477] INFO Established session 0x16427b3322603be with negotiated timeout 40000 for client /10.10.1.110:48847 (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:48.477608054Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,477] INFO Established session 0x16427b3322603bf with negotiated timeout 40000 for client /10.10.1.110:48850 (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:48.47790401Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,477] INFO Established session 0x16427b3322603c0 with negotiated timeout 40000 for client /10.10.1.110:48846 (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:48.498567248Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,498] INFO Processed session termination for sessionid: 0x16427b3322603bf (org.apache.zookeeper.server.PrepRequestProcessor)
2018-06-26T07:00:48.499902175Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,499] INFO Closed socket connection for client /10.10.1.110:48850 which had sessionid 0x16427b3322603bf (org.apache.zookeeper.server.NIOServerCnxn)
2018-06-26T07:00:48.50477847Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,504] INFO Accepted socket connection from /10.10.1.110:48852 (org.apache.zookeeper.server.NIOServerCnxnFactory)
2018-06-26T07:00:48.504997833Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,504] INFO Client attempting to establish new session at /10.10.1.110:48852 (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:48.505888207Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,505] INFO Established session 0x16427b3322603c1 with negotiated timeout 40000 for client /10.10.1.110:48852 (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:48.509890731Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,509] INFO Accepted socket connection from /10.10.1.110:48854 (org.apache.zookeeper.server.NIOServerCnxnFactory)
2018-06-26T07:00:48.510167901Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,510] INFO Client attempting to establish new session at /10.10.1.110:48854 (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:00:48.511113577Z mx-storm-zookeeper1 20 [2018-06-26 07:00:48,511] INFO Established session 0x16427b3322603c2 with negotiated timeout 40000 for client /10.10.1.110:48854 (org.apache.zookeeper.server.ZooKeeperServer)
2018-06-26T07:01:48.511113577Z 42438 Health is good
2018-06-26T07:01:58.511113577Z 42439 Health is good

0 个答案:

没有答案
相关问题
最新问题