在使用IgniteDataStreamer
插入数据时,有时会在地理空间部分遇到异常,因为它试图删除一行:
org.apache.ignite.internal.processors.query.h2.opt.GridH2SpatialIndex.remove(GridH2SpatialIndex.java:250) ~[ignite-geospatial-2.5.2.jar:2.6.0]
我使用单个数据库(一个具有一个数据库且所有Java应用程序为客户端的集群)进行了测试。
这是“部分”类:
public class LNSData implements Comparable<LNSData>{
@QuerySqlField(index = true)
public Long id;
...
@JsonIgnore
@QuerySqlField(index=true)
Geometry gwGeoCoordinates;
@JsonIgnore
@QuerySqlField(index=true)
Geometry gpsGeoCoordinates;
//constructor, getters, setters
}
两个几何都不应该为空,并且创建为:
WKTReader r = new WKTReader();
try {
gwGeoCoordinates = r.read("POINT(" + gatewayLong + " " + gatewayLat + " " + gatewayAlt +")");
} catch (ParseException e) {
LogManager.getLogger(LNSData.class).catching(e);
}
try {
gpsGeoCoordinates = r.read("POINT(" + gpsLongitude + " " + gpsLatitude + " " + gpsAltitude +")");
} catch (ParseException e) {
LogManager.getLogger(LNSData.class).catching(e);
}
这是流光部分:
TreeMap<Long, LNSData> dataToSave = new TreeMap<>();
try {
for (LNSData lnsData : lLNSData) {
dataToSave.put(lnsData.getId(), lnsData);
}
logger.debug("Calling repository save on " + lLNSData.size() + " elements");
if(stmr == null) {
stmr = databaseConfiguration.igniteInstance().dataStreamer("LNSDataCache");
stmr.autoFlushFrequency(2000);
}
stmr.addData(dataToSave);
}
最后是完整的错误:
[ERROR] 2018-08-01 08:48:30.316 [pool-787-thread-3] LNSDataDBManager - Catching
java.lang.IllegalStateException: Data streamer has been closed.
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.closedException(DataStreamerImpl.java:999) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.enterBusy(DataStreamerImpl.java:419) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:566) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:557) ~[ignite-core-2.6.0.jar:2.6.0]
at fr.kerlink.service.LNSDataDBManager.insertLNSDataEntries(LNSDataDBManager.java:86) [DataSurveillance.jar:3.4.0]
at fr.kerlink.business.ProcessData.run(ProcessData.java:134) [DataSurveillance.jar:3.4.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
Caused by: org.apache.ignite.IgniteCheckedException: Failed to finish operation (too many remaps): 32
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:881) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:846) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:347) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:335) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:495) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:462) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:125) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:45) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:347) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:335) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:495) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1923) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:346) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:511) ~[ignite-core-2.6.0.jar:2.6.0]
... 1 more
Caused by: org.apache.ignite.IgniteCheckedException: DataStreamer request failed [node=4da8e739-d686-42dd-b463-534abf84a7ca]
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1912) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:346) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:511) ~[ignite-core-2.6.0.jar:2.6.0]
... 1 more
Caused by: java.lang.RuntimeException: row not found
at org.h2.message.DbException.throwInternalError(DbException.java:242) ~[h2-1.4.195.jar:1.4.195]
at org.apache.ignite.internal.processors.query.h2.opt.GridH2SpatialIndex.remove(GridH2SpatialIndex.java:250) ~[ignite-geospatial-2.5.2.jar:2.6.0]
at org.apache.ignite.internal.processors.query.h2.opt.GridH2SpatialIndex.removex(GridH2SpatialIndex.java:263) ~[ignite-geospatial-2.5.2.jar:2.6.0]
at org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.addToIndex(GridH2Table.java:552) ~[ignite-indexing-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.update(GridH2Table.java:480) ~[ignite-indexing-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.store(IgniteH2Indexing.java:659) ~[ignite-indexing-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.query.GridQueryProcessor.store(GridQueryProcessor.java:1866) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.store(GridCacheQueryManager.java:403) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(IgniteCacheOffheapManagerImpl.java:1393) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1257) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:1529) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:352) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3602) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.initialValue(GridCacheMapEntry.java:2774) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:2125) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:140) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.localUpdate(DataStreamProcessor.java:400) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:305) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:60) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$1.onMessage(DataStreamProcessor.java:90) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091) ~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:511) ~[ignite-core-2.6.0.jar:2.6.0]
我最终丢失了很多数据。 感谢您的帮助。