我正在尝试使用Hazelcast连续查询缓存功能。
我的用例是跟踪分布式系统中的api调用总数,并在需要时应用阈值。每个客户端节点定期插入其自己的数据以进行映射并定期接收/查询 其他节点的数据。据我了解,连续查询缓存的作用类似于目标映射的本地/连续实例化视图,因此适合我的情况。
我的客户端服务器拓扑结构的hazelcast版本设置为3.9。我在客户端节点上使用连续查询缓存的合并功能。我正在使用ttl(通常为30秒)将元素插入地图。存在用于查询字段的索引。
这是我遇到的问题和问题:
1)当我尝试使用组合的Hazelcast谓词(EqualPredicate,GreaterPredicate等与and结合使用)进行连续缓存查询时,我经常在Hazelcast节点中看到以下异常:
2018-07-27 14:26:38,799 : [ERROR] [hz._hzInstance_1_mycluster.partition-operation.thread-7][SetOperation] [myhost]:9000 [mycluster] [3.9] null
java.lang.NullPointerException
at com.hazelcast.query.impl.CachedQueryEntry.getTargetObject(CachedQueryEntry.java:105)
at com.hazelcast.query.impl.QueryableEntry.extractAttributeValue(QueryableEntry.java:81)
at com.hazelcast.query.impl.QueryableEntry.getAttributeValue(QueryableEntry.java:48)
at com.hazelcast.query.impl.predicates.AbstractPredicate.readAttributeValue(AbstractPredicate.java:132)
at com.hazelcast.query.impl.predicates.AbstractPredicate.apply(AbstractPredicate.java:57)
at com.hazelcast.query.impl.predicates.NotEqualPredicate.apply(NotEqualPredicate.java:41)
at com.hazelcast.query.impl.predicates.AndPredicate.apply(AndPredicate.java:129)
at com.hazelcast.map.impl.query.QueryEventFilter.eval(QueryEventFilter.java:53)
at com.hazelcast.map.impl.event.AbstractFilteringStrategy.evaluateQueryEventFilter(AbstractFilteringStrategy.java:77)
at com.hazelcast.map.impl.event.DefaultEntryEventFilteringStrategy.processQueryEventFilter(DefaultEntryEventFilteringStrategy.java:125)
at com.hazelcast.map.impl.event.DefaultEntryEventFilteringStrategy.doFilter(DefaultEntryEventFilteringStrategy.java:84)
at com.hazelcast.map.impl.event.QueryCacheEventPublisher.getCQCEventTypeOrNull(QueryCacheEventPublisher.java:164)
at com.hazelcast.map.impl.event.QueryCacheEventPublisher.convertQueryCacheEventDataOrNull(QueryCacheEventPublisher.java:129)
at com.hazelcast.map.impl.event.QueryCacheEventPublisher.addEventToQueryCache(QueryCacheEventPublisher.java:92)
at com.hazelcast.map.impl.event.MapEventPublisherImpl.postPublishEvent(MapEventPublisherImpl.java:224)
at com.hazelcast.map.impl.event.MapEventPublisherImpl.publishEvent(MapEventPublisherImpl.java:204)
at com.hazelcast.map.impl.event.MapEventPublisherImpl.publishEvent(MapEventPublisherImpl.java:160)
at com.hazelcast.map.impl.event.MapEventPublisherImpl.publishEvent(MapEventPublisherImpl.java:148)
at com.hazelcast.map.impl.operation.BasePutOperation.afterRun(BasePutOperation.java:53)
at com.hazelcast.map.impl.operation.SetOperation.afterRun(SetOperation.java:40)
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.afterRun(OperationRunnerImpl.java:295)
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:196)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.run(OperationExecutorImpl.java:406)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.runOrExecute(OperationExecutorImpl.java:433)
at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeLocal(Invocation.java:569)
at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:554)
at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:513)
at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:207)
at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:60)
at com.hazelcast.client.impl.protocol.task.AbstractPartitionMessageTask.processMessage(AbstractPartitionMessageTask.java:67)
at com.hazelcast.client.impl.protocol.task.AbstractMessageTask.initializeAndProcessMessage(AbstractMessageTask.java:123)
at com.hazelcast.client.impl.protocol.task.AbstractMessageTask.run(AbstractMessageTask.java:103)
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:154)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:125)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.run(OperationThread.java:100)
这对我来说没有任何意义。看来,输入值是否为null?这是到期了吗?可以避免吗?
2)收到上述异常后,我实现了我的自定义谓词,如下所示:在应用谓词逻辑之前,我必须进行空检查
public class MyCustomPredicate implements Predicate<String, MyMapElement>, Serializable {
@Override
public boolean apply(Entry<String, MyMapElement> paramEntry) {
MyMapElement myMapElement = paramEntry.getValue();
if (myMapElement == null) {
return false;
}
//MY CUSTOM PREDICATE LOGIC
}
}
就像上面的输入值有时是空的……这就是为什么我不得不进行空检查的原因。
但是通过这种实现,我认为我的索引配置没有用吗?那正确吗?如果是这样,我如何使我的自定义谓词使用索引?
3)您会为我的问题推荐任何其他的hazelcast功能吗?在我看来,使用聚合的另一个选择是,但是我不确定哪个会更好。有什么建议吗?
4)这最终将在大约70个客户端(这意味着70个不同的连续查询)的生产代码中运行,并且地图大小约为50000。
最诚挚的问候