因此,我们在牧场主中拥有一个Neo4J实例,并使用docker image for 3.3。它有1GB的堆空间和页面缓存,但是由于这个问题,我们将其增加到2GB。
一切正常运行一段时间,直到我们开始收到此错误:
2018-08-02 06:03:30.310+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Check Pointing triggered by scheduler for time threshold [1942]: Starting check pointing...
2018-08-02 06:03:30.311+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Check Pointing triggered by scheduler for time threshold [1942]: Starting store flush...
2018-08-02 06:03:30.313+0000 ERROR [o.n.k.i.t.l.c.CheckPointerImpl] Error performing check point java.io.IOException: I/O error
org.neo4j.kernel.impl.store.UnderlyingStorageException: java.io.IOException: I/O error
at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:263)
at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce(RecordStorageEngine.java:460)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint(CheckPointerImpl.java:160)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded(CheckPointerImpl.java:134)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:64)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at org.neo4j.helpers.NamedThreadFactory$2.run(NamedThreadFactory.java:109)
Caused by: java.io.IOException: I/O error
at sun.nio.ch.FileDispatcherImpl.force0(Native Method)
at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:76)
at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:388)
at org.neo4j.io.fs.StoreFileChannel.force(StoreFileChannel.java:111)
at org.neo4j.io.pagecache.impl.SingleFilePageSwapper.force(SingleFilePageSwapper.java:711)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForceInternal(MuninnPagedFile.java:425)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce(MuninnPagedFile.java:274)
at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:901)
at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:894)
at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:259)
... 12 more
此错误多次发生,然后我们得到此错误:
2018-08-02 06:05:00.364+0000 ERROR [o.n.k.i.DatabaseHealth] Database panic: The database has encountered a critical error, and needs to be restarted. Please see database logs for more details. Error performing check point
org.neo4j.kernel.impl.store.UnderlyingStorageException: Error performing check point
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.constructCombinedFailure(CheckPointScheduler.java:100)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at org.neo4j.helpers.NamedThreadFactory$2.run(NamedThreadFactory.java:109)
Suppressed: org.neo4j.kernel.impl.store.UnderlyingStorageException: java.io.IOException: I/O error
at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:263)
at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce(RecordStorageEngine.java:460)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint(CheckPointerImpl.java:160)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded(CheckPointerImpl.java:134)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:64)
... 8 more
Caused by: java.io.IOException: I/O error
at sun.nio.ch.FileDispatcherImpl.force0(Native Method)
at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:76)
at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:388)
at org.neo4j.io.fs.StoreFileChannel.force(StoreFileChannel.java:111)
at org.neo4j.io.pagecache.impl.SingleFilePageSwapper.force(SingleFilePageSwapper.java:711)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForceInternal(MuninnPagedFile.java:425)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce(MuninnPagedFile.java:274)
at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:901)
at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:894)
at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:259)
... 12 more
Suppressed: org.neo4j.kernel.impl.store.UnderlyingStorageException: java.io.IOException: I/O error
at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:263)
at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce(RecordStorageEngine.java:460)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint(CheckPointerImpl.java:160)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded(CheckPointerImpl.java:134)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:64)
... 8 more
Caused by: java.io.IOException: I/O error
at sun.nio.ch.FileDispatcherImpl.force0(Native Method)
at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:76)
at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:388)
at org.neo4j.io.fs.StoreFileChannel.force(StoreFileChannel.java:111)
at org.neo4j.io.pagecache.impl.SingleFilePageSwapper.force(SingleFilePageSwapper.java:711)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForceInternal(MuninnPagedFile.java:425)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce(MuninnPagedFile.java:274)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce(MuninnPagedFile.java:262)
at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:921)
at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:894)
at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:259)
... 12 more
Suppressed: org.neo4j.kernel.impl.store.UnderlyingStorageException: java.io.IOException: I/O error
at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:263)
at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce(RecordStorageEngine.java:460)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint(CheckPointerImpl.java:160)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded(CheckPointerImpl.java:134)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:64)
... 8 more
Caused by: java.io.IOException: I/O error
at sun.nio.ch.FileDispatcherImpl.force0(Native Method)
at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:76)
at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:388)
at org.neo4j.io.fs.StoreFileChannel.force(StoreFileChannel.java:111)
at org.neo4j.io.pagecache.impl.SingleFilePageSwapper.force(SingleFilePageSwapper.java:711)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForceInternal(MuninnPagedFile.java:425)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce(MuninnPagedFile.java:274)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce(MuninnPagedFile.java:262)
at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:921)
at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:894)
at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:259)
... 12 more
Suppressed: org.neo4j.kernel.impl.store.UnderlyingStorageException: java.io.IOException: I/O error
at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:263)
at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce(RecordStorageEngine.java:460)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint(CheckPointerImpl.java:160)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded(CheckPointerImpl.java:134)
at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:64)
... 8 more
Caused by: java.io.IOException: I/O error
at sun.nio.ch.FileDispatcherImpl.force0(Native Method)
at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:76)
at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:388)
at org.neo4j.io.fs.StoreFileChannel.force(StoreFileChannel.java:111)
at org.neo4j.io.pagecache.impl.SingleFilePageSwapper.force(SingleFilePageSwapper.java:711)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForceInternal(MuninnPagedFile.java:425)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce(MuninnPagedFile.java:274)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce(MuninnPagedFile.java:262)
at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:921)
at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:894)
at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:259)
... 12 more
此后数据库无法访问,我们被迫重新启动它以使其开始工作。
这是在不一致的时间发生的,我们尚未找到对此或与之类似的帖子的明确原因。
有人曾经发生过这种情况吗?还是知道我能做些什么来阻止这种情况将来发生?