Question

我有一个缓存，它实际上将BinaryObject存储在一个集群中（2个节点）。 Ignite版本是2.1.0。如果我不使用任何StreamReceiver（包括StreamTransformer），使用以下代码添加大量BinaryObject数据时没有问题：

IgniteDataStreamer<Long,BinaryObject> ds = ignite.dataStreamer(CACHE_NAME);
SecureRandom random = new SecureRandom();
long i = 0;
long count = 1000000;
while(i++<count){
    builder.setField("id", i);
    builder.setField("name", "Test"+i);
    builder.setField("age", random.nextInt(30));
    builder.setField("score", random.nextDouble()*100d);
    builder.setField("birthday", new Date());
    ds.addData(i, builder.build());
    if(i%10000==0){
        System.out.println(i+" added...");
    }
}

但是现在，我想在添加之前修改我的BinaryObject数据值，所以我尝试了这样的StreamTransformer：

ds.receiver(new StreamTransformer<Long,BinaryObject>(){
    @Override
    public Object process(MutableEntry<Long, BinaryObject> entry, Object... arguments)
            throws EntryProcessorException {
        // TODO Auto-generated method stub
        Long key = entry.getKey();
        BinaryObject value = entry.getValue();
        BinaryObjectBuilder builder = value.toBuilder();
        //want to change the value of "name" field
        builder.setField("name", "Modify"+builder.getField("name"));
        entry.setValue(builder.build());
        return null;
    }
});
while(...){
    //... original code to build BinaryObject data and call ds.add method
}

不幸发生以下异常：

[09:52:36] Topology snapshot [ver=61, servers=2, clients=0, CPUs=8, heap=2.7GB]
10000 added...
20000 added...
30000 added...
40000 added...
[09:52:39,174][SEVERE][data-streamer-#54%null%][DataStreamerImpl] DataStreamer operation failed.
class org.apache.ignite.IgniteCheckedException: Failed to finish operation (too many remaps): 32
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:869)
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:834)
    at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:382)
    at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
    at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
    at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:494)
    at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:473)
    at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:461)
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer$2.apply(DataStreamerImpl.java:1572)
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer$2.apply(DataStreamerImpl.java:1562)
    at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:382)
    at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
    at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
    at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:494)
    at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:473)
    at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:461)
    at org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:967)
    at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.IgniteCheckedException: Unknown pair [platformId=0, typeId=-1496463502]
    at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7229)
    ... 5 more
Caused by: java.lang.ClassNotFoundException: Unknown pair [platformId=0, typeId=-1496463502]
    at org.apache.ignite.internal.MarshallerContextImpl.getClassName(MarshallerContextImpl.java:392)
    at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:342)
    at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:686)
    at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1755)
    at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
    at org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:797)
    at org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:143)
    at org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinary(CacheObjectUtils.java:161)
    at org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinaryIfNeeded(CacheObjectUtils.java:41)
    at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:125)
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerEntry$1.getValue(DataStreamerEntry.java:96)
    at org.apache.ignite.stream.StreamTransformer.receive(StreamTransformer.java:45)
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:137)
    at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6608)
    at org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:959)
    ... 4 more

我该怎么办才能修复它？

Answer 1

您得到$this->layout='app-main';因为ClassNotFoundException在内部尝试反序列化存储的DataStreamer。要使其直接使用BinaryObject，您应该在使用之前调用BinaryObject。

您的代码中的另一个问题是您使用ds.keepBinary(true)的结果的方式。实际上，传递给entry.getValue()方法的entry表示先前存储在缓存中的记录，因此您很可能在那里获得空值。如果要获取新分配的值，则应使用process值。

使用StreamTransformer for BinaryObject时出现“Unknown Pair”异常

1 个答案: