Apache Ignite-使用DataStreamer插入几何时找不到行

时间:2018-08-01 09:32:48

标签: java ignite

在使用IgniteDataStreamer插入数据时,有时会在地理空间部分遇到异常,因为它试图删除一行:

org.apache.ignite.internal.processors.query.h2.opt.GridH2SpatialIndex.remove(GridH2SpatialIndex.java:250) ~[ignite-geospatial-2.5.2.jar:2.6.0]

我使用单个数据库(一个具有一个数据库且所有Java应用程序为客户端的集群)进行了测试。

这是“部分”类:

public class LNSData implements Comparable<LNSData>{
    @QuerySqlField(index = true)
    public Long id;

    ...

    @JsonIgnore
    @QuerySqlField(index=true)
    Geometry gwGeoCoordinates;
    @JsonIgnore
    @QuerySqlField(index=true)
    Geometry gpsGeoCoordinates;

    //constructor, getters, setters
}

两个几何都不应该为空,并且创建为:

WKTReader r = new WKTReader();
try {
    gwGeoCoordinates = r.read("POINT(" + gatewayLong + " " + gatewayLat + " " + gatewayAlt +")");
} catch (ParseException e) {
    LogManager.getLogger(LNSData.class).catching(e);
}
try {
    gpsGeoCoordinates = r.read("POINT(" + gpsLongitude + " " + gpsLatitude + " " + gpsAltitude +")");
} catch (ParseException e) {
    LogManager.getLogger(LNSData.class).catching(e);
}

这是流光部分:

TreeMap<Long, LNSData> dataToSave = new TreeMap<>();            
try {    
    for (LNSData lnsData : lLNSData) {
        dataToSave.put(lnsData.getId(), lnsData);
    }

    logger.debug("Calling repository save on " + lLNSData.size() + " elements");

    if(stmr == null) {
        stmr = databaseConfiguration.igniteInstance().dataStreamer("LNSDataCache");
        stmr.autoFlushFrequency(2000);
    }
    stmr.addData(dataToSave);
}

最后是完整的错误:

[ERROR] 2018-08-01 08:48:30.316 [pool-787-thread-3] LNSDataDBManager - Catching
java.lang.IllegalStateException: Data streamer has been closed.
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.closedException(DataStreamerImpl.java:999) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.enterBusy(DataStreamerImpl.java:419) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:566) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:557) ~[ignite-core-2.6.0.jar:2.6.0]
    at fr.kerlink.service.LNSDataDBManager.insertLNSDataEntries(LNSDataDBManager.java:86) [DataSurveillance.jar:3.4.0]
    at fr.kerlink.business.ProcessData.run(ProcessData.java:134) [DataSurveillance.jar:3.4.0]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
Caused by: org.apache.ignite.IgniteCheckedException: Failed to finish operation (too many remaps): 32
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:881) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:846) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:347) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:335) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:495) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:462) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:125) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:45) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:347) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:335) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:495) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1923) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:346) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:511) ~[ignite-core-2.6.0.jar:2.6.0]
    ... 1 more
Caused by: org.apache.ignite.IgniteCheckedException: DataStreamer request failed [node=4da8e739-d686-42dd-b463-534abf84a7ca]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1912) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:346) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:511) ~[ignite-core-2.6.0.jar:2.6.0]
    ... 1 more
Caused by: java.lang.RuntimeException: row not found
    at org.h2.message.DbException.throwInternalError(DbException.java:242) ~[h2-1.4.195.jar:1.4.195]
    at org.apache.ignite.internal.processors.query.h2.opt.GridH2SpatialIndex.remove(GridH2SpatialIndex.java:250) ~[ignite-geospatial-2.5.2.jar:2.6.0]
    at org.apache.ignite.internal.processors.query.h2.opt.GridH2SpatialIndex.removex(GridH2SpatialIndex.java:263) ~[ignite-geospatial-2.5.2.jar:2.6.0]
    at org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.addToIndex(GridH2Table.java:552) ~[ignite-indexing-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.update(GridH2Table.java:480) ~[ignite-indexing-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.store(IgniteH2Indexing.java:659) ~[ignite-indexing-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.store(GridQueryProcessor.java:1866) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.store(GridCacheQueryManager.java:403) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(IgniteCacheOffheapManagerImpl.java:1393) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1257) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:1529) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:352) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3602) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.initialValue(GridCacheMapEntry.java:2774) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:2125) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:140) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.localUpdate(DataStreamProcessor.java:400) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:305) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:60) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$1.onMessage(DataStreamProcessor.java:90) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091) ~[ignite-core-2.6.0.jar:2.6.0]
    at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:511) ~[ignite-core-2.6.0.jar:2.6.0]

我最终丢失了很多数据。 感谢您的帮助。

0 个答案:

没有答案