我正在开发一个拥有数百万个节点和关系的大型项目。我最终想要的是“只是”加快数据输入的方法。
我已经意识到它真的很慢,只能在一个CPU上运行。据我所知,没有什么可以做的,我在这里读到的一件事:Neo4j cpu stuck on GC是GC可能是个问题。
我试过:“wrapper.java.additional = -XX:+ UseConcMarkSweepGC”和“wrapper.java.additional = -XX:+ UseG1GC”垃圾收集器。但是,任何人都可以解释我的差异,以及我应该使用哪一个(或其他)来加速向neo4j输入数据?
G1日志问题:
2015-03-03T16:03:09.739+0000: 165.539: Total time for which application threads were stopped: 0.0002486 seconds
2015-03-03T16:03:10.377+0000: 166.177: [GC pause (G1 Evacuation Pause) (young)
Desired survivor size 322961408 bytes, new threshold 15 (max 15)
- age 1: 582688 bytes, 582688 total
- age 2: 570272 bytes, 1152960 total
- age 3: 559776 bytes, 1712736 total
- age 4: 636744 bytes, 2349480 total
- age 5: 530944 bytes, 2880424 total
- age 6: 608416 bytes, 3488840 total
- age 7: 497416 bytes, 3986256 total
- age 8: 541248 bytes, 4527504 total
- age 9: 533328 bytes, 5060832 total
- age 10: 638496 bytes, 5699328 total
- age 11: 522696 bytes, 6222024 total
- age 12: 456152 bytes, 6678176 total
- age 13: 623536 bytes, 7301712 total
- age 14: 503232 bytes, 7804944 total
- age 15: 520504 bytes, 8325448 total
, 0.0171905 secs]
[Parallel Time: 14.0 ms, GC Workers: 4]
[GC Worker Start (ms): Min: 166177.2, Avg: 166178.1, Max: 166180.5, Diff: 3.3]
[Ext Root Scanning (ms): Min: 0.0, Avg: 0.6, Max: 1.2, Diff: 1.2, Sum: 2.4]
[Update RS (ms): Min: 0.0, Avg: 1.9, Max: 2.7, Diff: 2.7, Sum: 7.5]
[Processed Buffers: Min: 4, Avg: 5.2, Max: 6, Diff: 2, Sum: 21]
[Scan RS (ms): Min: 0.4, Avg: 0.6, Max: 0.8, Diff: 0.4, Sum: 2.5]
[Code Root Scanning (ms): Min: 0.1, Avg: 0.1, Max: 0.1, Diff: 0.1, Sum: 0.3]
[Object Copy (ms): Min: 9.6, Avg: 9.8, Max: 10.3, Diff: 0.7, Sum: 39.2]
[Termination (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[GC Worker Other (ms): Min: 0.0, Avg: 0.1, Max: 0.1, Diff: 0.1, Sum: 0.3]
[GC Worker Total (ms): Min: 10.6, Avg: 13.1, Max: 13.9, Diff: 3.3, Sum: 52.2]
[GC Worker End (ms): Min: 166191.1, Avg: 166191.1, Max: 166191.1, Diff: 0.1]
[Code Root Fixup: 0.1 ms]
[Code Root Migration: 0.3 ms]
[Code Root Purge: 0.0 ms]
[Clear CT: 0.8 ms]
[Other: 2.0 ms]
[Choose CSet: 0.0 ms]
[Ref Proc: 0.4 ms]
[Ref Enq: 0.0 ms]
[Redirty Cards: 0.1 ms]
[Free CSet: 1.3 ms]
[Eden: 4900.0M(4900.0M)->0.0B(4896.0M) Survivors: 12.0M->16.0M Heap: 5018.7M(8192.0M)->121.8M(8192.0M)]
[Times: user=0.06 sys=0.00, real=0.02 secs]
CMS问题:
2015-03-03T15:32:51.670+0000: 240.063: [GC (Allocation Failure) 240.063: [ParNew
Desired survivor size 17432576 bytes, new threshold 6 (max 6)
- age 1: 103000 bytes, 103000 total
- age 2: 20400 bytes, 123400 total
- age 3: 27496 bytes, 150896 total
- age 4: 22968 bytes, 173864 total
- age 5: 62400 bytes, 236264 total
- age 6: 23960 bytes, 260224 total
: 272985K->408K(306688K), 0.0245535 secs] 390715K->118161K(8354560K), 0.0246580 secs] [Times: user=0.09 sys=0.00, real=0.03 secs]
奖励信息:
CONF / neo4j.property:
# Default values for the low-level graph engine
neostore.nodestore.db.mapped_memory=4G
#neostore.relationshipstore.db.mapped_memory=50M
#neostore.relationshipgroupstore.db.mapped_memory=10M
#neostore.propertystore.db.mapped_memory=90M
#neostore.propertystore.db.strings.mapped_memory=130M
#neostore.propertystore.db.arrays.mapped_memory=130M
CONF / Neo4j的-wrapper.conf:
wrapper.java.additional=-Dorg.neo4j.server.properties=conf/neo4j-server.properties
wrapper.java.additional=-Djava.util.logging.config.file=conf/logging.properties
wrapper.java.additional=-Dlog4j.configuration=file:conf/log4j.properties
#********************************************************************
# JVM Parameters
#********************************************************************
#wrapper.java.additional=-XX:+UseConcMarkSweepGC
wrapper.java.additional=-XX:+CMSClassUnloadingEnabled
wrapper.java.additional=-XX:-OmitStackTraceInFastThrow
# Own GC
wrapper.java.additional=-XX:+UseG1GC
# Uncomment the following lines to enable garbage collection logging
wrapper.java.additional=-Xloggc:data/log/neo4j-gc.log
wrapper.java.additional=-XX:+PrintGCDetails
wrapper.java.additional=-XX:+PrintGCDateStamps
wrapper.java.additional=-XX:+PrintGCApplicationStoppedTime
wrapper.java.additional=-XX:+PrintPromotionFailure
wrapper.java.additional=-XX:+PrintTenuringDistribution
# Java Heap Size: by default the Java heap size is dynamically
# calculated based on available system resources.
# Uncomment these lines to set specific initial and maximum
# heap size in MB.
wrapper.java.initmemory=8192
wrapper.java.maxmemory=8192
#********************************************************************
# Wrapper settings
#********************************************************************
# path is relative to the bin dir
wrapper.pidfile=../data/neo4j-server.pid
#********************************************************************
# Wrapper Windows NT/2000/XP Service Properties
#********************************************************************
# WARNING - Do not modify any of these properties when an application
# using this configuration file has been installed as a service.
# Please uninstall the service before modifying this section. The
# service can then be reinstalled.
# Name of the service
wrapper.name=neo4j
# User account to be used for linux installs. Will default to current
# user if not se 1,1