Question

我使用JHipster创建了一个单片应用程序，我想在GCP Kubernetes Engine集群上运行它的几个副本。

由于我想在副本（甚至是该群集上的服务）之间共享数据，我决定使用内置的Hazelcast缓存，而不是使用我想要使用的推荐的Eureka发现客户端K8 dns-discovery附加组件。

所以我已经添加到我的pom.xml中了：

<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast-kubernetes</artifactId>
    <version>1.1.0</version>
</dependency>

并使用生成的CacheConfiguration.java文件稍作支持hazelcast-kubernetes配置（基于他们的github repo）：

@Bean
public HazelcastInstance hazelcastInstance(JHipsterProperties jHipsterProperties) {
    log.debug("Configuring Hazelcast");
    HazelcastInstance hazelCastInstance = Hazelcast.getHazelcastInstanceByName("hazelcastK8");
    if (hazelCastInstance != null) {
        log.debug("Hazelcast already initialized");
        return hazelCastInstance;
    }
    Config config = new Config();
    config.setInstanceName("hazelcastK8");
    config.getNetworkConfig().setPort(5701);
    config.getNetworkConfig().setPortAutoIncrement(true);

    config.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false);
    config.getNetworkConfig().getJoin().getTcpIpConfig().setEnabled(false);

    if (env.acceptsProfiles(JHipsterConstants.SPRING_PROFILE_DEVELOPMENT)) {
        System.setProperty("hazelcast.local.localAddress", "127.0.0.1");
        config.getNetworkConfig().getJoin().getAwsConfig().setEnabled(false);
    } else { // In production we want to use hazelcast k8's dns discovery service
        log.info("Configuring hazelcast DNS discovery by K8");

        DiscoveryStrategyConfig strategyConfig =
            new DiscoveryStrategyConfig(
                new HazelcastKubernetesDiscoveryStrategyFactory()
            );

        // Default namespace
        strategyConfig.addProperty("service-dns", "hazelcastk8.default.svc.cluster.local");
        strategyConfig.addProperty("service-dns-timeout", 10);

        config.getNetworkConfig().getJoin().getDiscoveryConfig().addDiscoveryStrategyConfig(strategyConfig);
    }

    config.getMapConfigs().put("default", initializeDefaultMapConfig());
    config.setManagementCenterConfig(initializeDefaultManagementCenterConfig(jHipsterProperties));
    config.getMapConfigs().put("com.company.distributed.domain.*", initializeDomainMapConfig(jHipsterProperties));

    return Hazelcast.newHazelcastInstance(config);
}

为了检查数据网格是否按预期工作，我编写了一个简单的计划任务，打印出原子序列的当前值：

@Service
@Profile(JHipsterConstants.SPRING_PROFILE_PRODUCTION)
public class FetchHazelcastClusterSequence {

    private final HazelcastInstance hazelcastInstance;

    private static final Logger log = LoggerFactory.getLogger(FetchHazelcastClusterSequence.class);

    public FetchHazelcastClusterSequence(HazelcastInstance hazelcastInstance) {
        this.hazelcastInstance = hazelcastInstance;
    }

    @Timed
    @Scheduled(fixedDelay = 5000)
    public void start() {
        log.info("Fetching cluster sequence, the current value is: {}", this.hazelcastInstance.getAtomicLong("sequence").getAndIncrement());
    }
}

但是当我启动应用程序时，默认配置为3个副本，创建了3个pod，但每个pod日志显示为它是集群中唯一的成员，因此打印相同的“序列”值”

我错过了一些配置吗？

Answer 1

@ lior-ziv，您的配置缺少一个重要的配置参数：

1 - 使用SPRING_PROFILE_DEVELOPMENT个人资料时，您没有启用任何发现机制。您需要启用TCP-IP发现，否则所有成员都将启动单个节点集群。

2 - 使用SPRING_PROFILE_PRODUCTION个人资料时，您拥有Kubernetes发现插件，但您根本没有启用发现。这就是为什么每次启动新实例时它也会创建单个成员集群。

正确的配置应该是这样的：

//Move this like outside of if block since it'll be disabled for both cases.
config.getNetworkConfig().getJoin().getAwsConfig().setEnabled(false);

if (env.acceptsProfiles(JHipsterConstants.SPRING_PROFILE_DEVELOPMENT)) {
    config.getNetworkConfig().getJoin().getTcpIpConfig().setEnabled(true);
    config.getNetworkConfig().getJoin().getTcpIpConfig().addMember("127.0.0.1");`
} else {
    // In production we want to use hazelcast k8's dns discovery service
    System.setProperty("hazelcast.discovery.enabled", "true");
    log.info("Configuring hazelcast DNS discovery by K8");

    DiscoveryStrategyConfig strategyConfig =
        new DiscoveryStrategyConfig(
            new HazelcastKubernetesDiscoveryStrategyFactory()
        );

    // Default namespace
    strategyConfig.addProperty("service-dns", "hazelcastk8.default.svc.cluster.local");
    strategyConfig.addProperty("service-dns-timeout", 10);

    config.getNetworkConfig().getJoin().getDiscoveryConfig().addDiscoveryStrategyConfig(strategyConfig);
}

使用Kubernetes dns附加组件发现多个Hazelcast实例的问题

1 个答案: