如何使Apache Ignite持久性工作?

时间:2017-12-06 02:30:32

标签: persistence ignite

我的嵌入式Ignite(v2.3)服务具有以下配置(最终它将成为集群,但嵌入式更易于开发):

    val config = new IgniteConfiguration

    val dsConfig = new DataStorageConfiguration
    dsConfig.setStoragePath("ignitedb/storage")
    dsConfig.setWalPath("ignitedb/wal")
    dsConfig.setWalArchivePath("ignitedb/wal/archive")
    dsConfig.getDefaultDataRegionConfiguration.setPersistenceEnabled(true)
    config.setDataStorageConfiguration(dsConfig)

    Ignition.setClientMode(false)
    config.setConsistentId(java.net.InetAddress.getLocalHost.getHostName)

    val ignite = Ignition.start(config)
    ignite.active(true)

在这些目录中创建了各种文件:

ignitedb/
├── storage
│   └── imac_home
│       ├── cache-changes
│       │   ├── 1113bbb3-6183-4f97-8783-ca61c4afffb0
│       │   │   ├── cache_data.dat
│       │   │   └── index.bin
│       │   └── 605e79c8-63aa-49d5-86b2-68468e648375
│       │       ├── cache_data.dat
│       │       └── index.bin
│       ├── cache-ignite-sys-cache
│       ├── cache-pings
│       │   ├── 86d44085-3eca-4dfa-92fb-c2e7fd0f4f4b
│       │   │   ├── cache_data.dat
│       │   │   ├── index.bin
│       │   │   └── part-714.bin
│       │   └── f69da997-fae5-45a5-9abc-44639bb46e8e
│       │       ├── cache_data.dat
│       │       ├── index.bin
│       │       ├── part-142.bin
│       │       ├── part-152.bin
│       │       ├── <many more>
│       ├── cache-products
│       │   ├── 1113bbb3-6183-4f97-8783-ca61c4afffb0
│       │   │   ├── cache_data.dat
│       │   │   ├── index.bin
│       │   │   ├── part-49.bin
│       │   │   ├── part-50.bin
│       │   │   ├── <many more>
│       │   └── 605e79c8-63aa-49d5-86b2-68468e648375
│       │       ├── cache_data.dat
│       │       ├── index.bin
│       │       └── part-49.bin
│       ├── cacheGroup-default-ds-group
│       │   ├── ignite-sys-atomic-cache@default-ds-groupcache_data.dat
│       │   ├── index.bin
│       │   ├── part-264.bin
│       │   └── part-791.bin
│       ├── cp
│       │   ├── 1512524465868-cecd4b2e-5c0f-4b25-8ede-92336ba2aac3-END.bin
│       │   ├── 1512524465868-cecd4b2e-5c0f-4b25-8ede-92336ba2aac3-START.bin
│       │   ├── 1512524645873-70f73379-5e49-4779-849c-e1dda08e2f84-END.bin
│       │   ├── 1512524645873-70f73379-5e49-4779-849c-e1dda08e2f84-START.bin
│       │   ├── <many more>
│       └── lock
└── wal
    ├── archive
    │   └── imac_home
    └── imac_home
        ├── 0000000000000000.wal
        ├── 0000000000000001.wal
        ├── <many more>

然而,当我停止并重启我的应用程序时,我的所有缓存都消失了。我如何实际将它们保存到磁盘?我是否需要为每个缓存执行某些操作以及在全局范围内启用持久性? (文档中没有提到类似的内容。)

我在启动时遇到这个异常,经过仔细检查看起来它来自持久层:

09:44:33.034 INFO  o.a.i.i.p.c.p.f.FilePageStoreManager - Resolved page store work directory: /Users/dan/projects/livefeeds/ignitedb/storage/imac_home
09:44:33.034 INFO  o.a.i.i.p.c.p.w.FileWriteAheadLogManager - Resolved write ahead log work directory: /Users/dan/projects/livefeeds/ignitedb/wal/imac_home
09:44:33.034 INFO  o.a.i.i.p.c.p.w.FileWriteAheadLogManager - Resolved write ahead log archive directory: /Users/dan/projects/livefeeds/ignitedb/wal/archive/imac_home
09:44:33.044 INFO  o.a.i.i.p.c.p.w.FileWriteAheadLogManager - Started write-ahead log manager [mode=DEFAULT]
Can't load log handler "org.apache.ignite.logger.java.JavaLoggerFileHandler"
java.lang.ClassNotFoundException: org.apache.ignite.logger.java.JavaLoggerFileHandler
java.lang.ClassNotFoundException: org.apache.ignite.logger.java.JavaLoggerFileHandler
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.util.logging.LogManager$5.run(LogManager.java:965)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.util.logging.LogManager.loadLoggerHandlers(LogManager.java:958)
    at java.util.logging.LogManager.initializeGlobalHandlers(LogManager.java:1578)
    at java.util.logging.LogManager.access$1500(LogManager.java:145)
    at java.util.logging.LogManager$RootLogger.accessCheckedHandlers(LogManager.java:1667)
    at java.util.logging.Logger.getHandlers(Logger.java:1776)
    at org.apache.ignite.logger.java.JavaLogger.findHandler(JavaLogger.java:399)
    at org.apache.ignite.logger.java.JavaLogger.configure(JavaLogger.java:229)
    at org.apache.ignite.logger.java.JavaLogger.<init>(JavaLogger.java:170)
    at org.apache.ignite.logger.java.JavaLogger.<init>(JavaLogger.java:126)
    at org.apache.ignite.IgniteJdbcDriver.<clinit>(IgniteJdbcDriver.java:410)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
    at java.lang.Class.newInstance(Class.java:442)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
    at java.util.ServiceLoader$LazyIterator.access$700(ServiceLoader.java:323)
    at java.util.ServiceLoader$LazyIterator$2.run(ServiceLoader.java:407)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:409)
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
    at java.sql.DriverManager$2.run(DriverManager.java:603)
    at java.sql.DriverManager$2.run(DriverManager.java:583)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.sql.DriverManager.loadInitialDrivers(DriverManager.java:583)
    at java.sql.DriverManager.<clinit>(DriverManager.java:101)
    at org.h2.Driver.load(Driver.java:155)
    at org.h2.Driver.<clinit>(Driver.java:41)
    at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.start(IgniteH2Indexing.java:2027)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.start(GridQueryProcessor.java:240)
    at org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1870)
    at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:955)
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1909)
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1652)
    at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1080)
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:600)
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:525)
    at org.apache.ignite.Ignition.start(Ignition.java:322)
    at io.livefeeds.api.Main$.startIgnite(Main.scala:78)
    at io.livefeeds.api.Main$.main(Main.scala:25)
    at io.livefeeds.api.Main.main(Main.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at sbt.Run.invokeMain(Run.scala:67)
    at sbt.Run.run0(Run.scala:61)
    at sbt.Run.sbt$Run$$execute$1(Run.scala:51)
    at sbt.Run$$anonfun$run$1.apply$mcV$sp(Run.scala:55)
    at sbt.Run$$anonfun$run$1.apply(Run.scala:55)
    at sbt.Run$$anonfun$run$1.apply(Run.scala:55)
    at sbt.Logger$$anon$4.apply(Logger.scala:84)
    at sbt.TrapExit$App.run(TrapExit.scala:248)
    at java.lang.Thread.run(Thread.java:745)

持久性的其余日志输出是:

09:44:34.682 INFO  o.a.i.i.p.c.p.f.FilePageStoreManager - Resolved page store work directory: /Users/dan/projects/livefeeds/ignitedb/storage/imac_home
09:44:34.682 INFO  o.a.i.i.p.c.p.w.FileWriteAheadLogManager - Resolved write ahead log work directory: /Users/dan/projects/livefeeds/ignitedb/wal/imac_home
09:44:34.682 INFO  o.a.i.i.p.c.p.w.FileWriteAheadLogManager - Resolved write ahead log archive directory: /Users/dan/projects/livefeeds/ignitedb/wal/archive/imac_home
09:44:34.682 INFO  o.a.i.i.p.c.p.w.FileWriteAheadLogManager - Started write-ahead log manager [mode=DEFAULT]
09:44:34.747 INFO  o.a.i.i.p.c.p.p.PageMemoryImpl - Started page memory [memoryAllocated=6.4 GiB, pages=1634272, tableSize=94.9 MiB, checkpointBuffer=1.6 GiB]
09:44:34.785 INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager - Read checkpoint status [startMarker=/Users/dan/projects/livefeeds/ignitedb/storage/imac_home/cp/1512571436282-f25aee22-0e9e-4737-a0ad-aebc3c5a4bd6-START.bin, endMarker=/Users/dan/projects/livefeeds/ignitedb/storage/imac_home/cp/1512571307494-690bb0bf-e75e-4877-a3e5-ba7b43b190bd-END.bin]
09:44:34.786 INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager - Checking memory state [lastValidPos=FileWALPointer [idx=0, fileOffset=8996738, len=20281, forceFlush=false], lastMarked=FileWALPointer [idx=0, fileOffset=9076734, len=20281, forceFlush=false], lastCheckpointId=f25aee22-0e9e-4737-a0ad-aebc3c5a4bd6]
09:44:34.786 WARN  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager - Ignite node stopped in the middle of checkpoint. Will restore memory state and finish checkpoint on node start.
[09:44:34] Ignite node stopped in the middle of checkpoint. Will restore memory state and finish checkpoint on node start.
09:44:34.846 INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager - Found last checkpoint marker [cpId=f25aee22-0e9e-4737-a0ad-aebc3c5a4bd6, pos=FileWALPointer [idx=0, fileOffset=9076734, len=20281, forceFlush=false]]
09:44:34.853 INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager - Finished applying memory changes [changesApplied=8, time=65ms]
09:44:34.857 INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager - Checkpoint finished [cpId=f25aee22-0e9e-4737-a0ad-aebc3c5a4bd6, pages=6, markPos=FileWALPointer [idx=0, fileOffset=9076734, len=20281, forceFlush=false], pagesWrite=-3ms, fsync=0ms, total=-3ms]
09:44:34.861 INFO  o.a.i.i.p.c.p.w.FileWriteAheadLogManager - Resuming logging to WAL segment [file=/Users/dan/projects/livefeeds/ignitedb/wal/imac_home/0000000000000000.wal, offset=9097015, ver=1]
09:44:35.065 INFO  o.a.i.i.p.cache.GridCacheProcessor - Started cache [name=ignite-sys-cache, memoryPolicyName=sysMemPlc, mode=REPLICATED, atomicity=TRANSACTIONAL]
09:44:35.085 INFO  o.a.i.i.p.cache.GridCacheProcessor - Started cache [name=ignite-sys-atomic-cache@default-ds-group, group=default-ds-group, memoryPolicyName=default, mode=PARTITIONED, atomicity=TRANSACTIONAL]
09:44:35.087 INFO  o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture - Successfully activated caches [nodeId=f616a68d-e171-4384-8dc4-45c2e6788afc, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
09:44:35.091 INFO  o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture - Finished waiting for partition release future [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], waitTime=0ms, futInfo=NA]
09:44:35.139 INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager - Read checkpoint status [startMarker=/Users/dan/projects/livefeeds/ignitedb/storage/imac_home/cp/1512571436282-f25aee22-0e9e-4737-a0ad-aebc3c5a4bd6-START.bin, endMarker=/Users/dan/projects/livefeeds/ignitedb/storage/imac_home/cp/1512571436282-f25aee22-0e9e-4737-a0ad-aebc3c5a4bd6-END.bin]
09:44:35.140 INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager - Applying lost cache updates since last checkpoint record [lastMarked=FileWALPointer [idx=0, fileOffset=9076734, len=20281, forceFlush=false], lastCheckpointId=f25aee22-0e9e-4737-a0ad-aebc3c5a4bd6]
09:44:35.187 INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager - Finished applying WAL changes [updatesApplied=0, time=51ms]
09:44:35.189 INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager - Finished indexes rebuilding for cache: [name=ignite-sys-cache, grpName=null

1 个答案:

答案 0 :(得分:1)

问题是我的缓存名称中的/导致持久层创建了额外的目录,因为它们根本没有被转义。我转而使用@,现在看起来很好。