重新加载更新的Fat-jar时Vertx hazelcast群集不稳定

时间:2019-02-03 05:34:53

标签: java hazelcast vert.x

我正在使用群集的Vertx(vertx 3.6.0和vertx-hazelcast 3.6.2)进行一些实验。 目前,我实现了一个玩具分布式注册表{String => String},该注册表在由HTTP触发的具有基本操作(setgetdel)的几个节点上运行。例如,curl http://localhost:8081/set/key1/val1请求侦听端口8081的节点将值val1存储在键var1下。注册表的数据存储区是Vertx通过执行vertx.sharedData().<String, String>getAsyncMap(...)提供的AsyncMap。到目前为止,我没有配置与Hazelcast相关的任何内容,因此,我使用的是该库随附的默认配置。节点故障和恢复,添加新节点,同时添加多个节点等测试用例都可以正常工作。

我的测试用例之一是没有停机的应用程序更新。如下:

  1. 在HTTP负载均衡器后面运行一些节点,这些请求将在节点上分发请求
  2. 更新应用程序(添加一些功能并重新编译应用程序。为示例起见,我实现了keys函数,该函数返回注册表中定义的键列表)。
  3. 从负载均衡器中删除node_1并停止它。
  4. 使用新的fat-jar重新启动node_1,让它加入集群并重新同步映射,然后在负载均衡器中重新启用它。
  5. 对所有节点执行相同操作。
  6. 使客户端能够使用新功能。

问题发生在第4步:更新后的节点大部分时间都在重新启动,而没有记录任何错误,但是在尚未更新的节点上我得到了错误和异常,如果发生这种情况,则新节点上的映射就不​​会同步(空),旧节点不再响应HTTP请求。这不是系统性的,有时一切都很好,但是在大多数情况下,繁荣!

例如:

févr. 03, 2019 7:56:54 AM com.hazelcast.internal.partition.operation.MigrationRequestOperation
WARNING: [192.168.8.149]:5701 [dev] [3.10.5] Error while executing beforeMigration()
java.lang.NoClassDefFoundError: com/hazelcast/map/impl/querycache/publisher/AccumulatorSweeper
    at com.hazelcast.map.impl.MapMigrationAwareService.flushAndRemoveQueryCaches(MapMigrationAwareService.java:104)
    at com.hazelcast.map.impl.MapMigrationAwareService.beforeMigration(MapMigrationAwareService.java:88)
    at com.hazelcast.spi.impl.CountingMigrationAwareService.beforeMigration(CountingMigrationAwareService.java:76)
    at com.hazelcast.map.impl.MapService.beforeMigration(MapService.java:147)
    at com.hazelcast.internal.partition.operation.BaseMigrationOperation.executeBeforeMigrations(BaseMigrationOperation.java:180)
    at com.hazelcast.internal.partition.operation.MigrationRequestOperation.executeBeforeMigrations(MigrationRequestOperation.java:186)
    at com.hazelcast.internal.partition.operation.MigrationRequestOperation.run(MigrationRequestOperation.java:216)
    at com.hazelcast.spi.Operation.call(Operation.java:148)
    at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:202)
    at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:191)
    at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:405)
    at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:115)
    at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.run(OperationThread.java:100)

使用相同的源代码执行相同的过程时得到的另一个示例:

févr. 03, 2019 8:24:30 AM com.hazelcast.nio.tcp.TcpIpConnectionManager
INFO: [192.168.8.149]:5701 [dev] [3.10.5] Established socket connection between /192.168.8.149:5701 and /192.168.8.149:34634
févr. 03, 2019 8:24:32 AM com.hazelcast.internal.networking.nio.iobalancer.IOBalancer
SEVERE: [192.168.8.149]:5701 [dev] [3.10.5] IOBalancer failed
java.lang.NoClassDefFoundError: com/hazelcast/internal/networking/nio/NioInboundPipeline$StartMigrationTask
    at com.hazelcast.internal.networking.nio.NioInboundPipeline.requestMigration(NioInboundPipeline.java:115)
    at com.hazelcast.internal.networking.nio.iobalancer.IOBalancer.tryMigrate(IOBalancer.java:212)
    at com.hazelcast.internal.networking.nio.iobalancer.IOBalancer.scheduleMigrationIfNeeded(IOBalancer.java:153)
    at com.hazelcast.internal.networking.nio.iobalancer.IOBalancer.checkInboundPipelines(IOBalancer.java:146)
    at com.hazelcast.internal.networking.nio.iobalancer.IOBalancerThread.run(IOBalancerThread.java:50)
Caused by: java.lang.ClassNotFoundException: com.hazelcast.internal.networking.nio.NioInboundPipeline$StartMigrationTask
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    ... 5 more

有什么想法吗?

0 个答案:

没有答案