Question

我正在开发一个拥有数百万个节点和关系的大型项目。我最终想要的是“只是”加快数据输入的方法。

我已经意识到它真的很慢，只能在一个CPU上运行。据我所知，没有什么可以做的，我在这里读到的一件事：Neo4j cpu stuck on GC是GC可能是个问题。

我试过：“wrapper.java.additional = -XX：+ UseConcMarkSweepGC”和“wrapper.java.additional = -XX：+ UseG1GC”垃圾收集器。但是，任何人都可以解释我的差异，以及我应该使用哪一个（或其他）来加速向neo4j输入数据？

G1日志问题：

2015-03-03T16:03:09.739+0000: 165.539: Total time for which application threads were stopped: 0.0002486 seconds
2015-03-03T16:03:10.377+0000: 166.177: [GC pause (G1 Evacuation Pause) (young)
Desired survivor size 322961408 bytes, new threshold 15 (max 15)
- age   1:     582688 bytes,     582688 total
- age   2:     570272 bytes,    1152960 total
- age   3:     559776 bytes,    1712736 total
- age   4:     636744 bytes,    2349480 total
- age   5:     530944 bytes,    2880424 total
- age   6:     608416 bytes,    3488840 total
- age   7:     497416 bytes,    3986256 total
- age   8:     541248 bytes,    4527504 total
- age   9:     533328 bytes,    5060832 total
- age  10:     638496 bytes,    5699328 total
- age  11:     522696 bytes,    6222024 total
- age  12:     456152 bytes,    6678176 total
- age  13:     623536 bytes,    7301712 total
- age  14:     503232 bytes,    7804944 total
- age  15:     520504 bytes,    8325448 total
, 0.0171905 secs]
   [Parallel Time: 14.0 ms, GC Workers: 4]
      [GC Worker Start (ms): Min: 166177.2, Avg: 166178.1, Max: 166180.5, Diff: 3.3]
      [Ext Root Scanning (ms): Min: 0.0, Avg: 0.6, Max: 1.2, Diff: 1.2, Sum: 2.4]
      [Update RS (ms): Min: 0.0, Avg: 1.9, Max: 2.7, Diff: 2.7, Sum: 7.5]
         [Processed Buffers: Min: 4, Avg: 5.2, Max: 6, Diff: 2, Sum: 21]
      [Scan RS (ms): Min: 0.4, Avg: 0.6, Max: 0.8, Diff: 0.4, Sum: 2.5]
      [Code Root Scanning (ms): Min: 0.1, Avg: 0.1, Max: 0.1, Diff: 0.1, Sum: 0.3]
      [Object Copy (ms): Min: 9.6, Avg: 9.8, Max: 10.3, Diff: 0.7, Sum: 39.2]
      [Termination (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
      [GC Worker Other (ms): Min: 0.0, Avg: 0.1, Max: 0.1, Diff: 0.1, Sum: 0.3]
      [GC Worker Total (ms): Min: 10.6, Avg: 13.1, Max: 13.9, Diff: 3.3, Sum: 52.2]
      [GC Worker End (ms): Min: 166191.1, Avg: 166191.1, Max: 166191.1, Diff: 0.1]
   [Code Root Fixup: 0.1 ms]
   [Code Root Migration: 0.3 ms]
   [Code Root Purge: 0.0 ms]
   [Clear CT: 0.8 ms]
   [Other: 2.0 ms]
      [Choose CSet: 0.0 ms]
      [Ref Proc: 0.4 ms]
      [Ref Enq: 0.0 ms]
      [Redirty Cards: 0.1 ms]
      [Free CSet: 1.3 ms]
   [Eden: 4900.0M(4900.0M)->0.0B(4896.0M) Survivors: 12.0M->16.0M Heap: 5018.7M(8192.0M)->121.8M(8192.0M)]
 [Times: user=0.06 sys=0.00, real=0.02 secs]

CMS问题：

2015-03-03T15:32:51.670+0000: 240.063: [GC (Allocation Failure) 240.063: [ParNew
Desired survivor size 17432576 bytes, new threshold 6 (max 6)
- age 1: 103000 bytes, 103000 total
- age 2: 20400 bytes, 123400 total
- age 3: 27496 bytes, 150896 total
- age 4: 22968 bytes, 173864 total
- age 5: 62400 bytes, 236264 total
- age 6: 23960 bytes, 260224 total
: 272985K->408K(306688K), 0.0245535 secs] 390715K->118161K(8354560K), 0.0246580 secs] [Times: user=0.09 sys=0.00, real=0.03 secs]

奖励信息：

我使用的是neo4j-community-2.1.7
我在AWS M3.xlarge服务器上工作
使用py2neo输入数据。
我已将max file-limit设置为40k

CONF / neo4j.property：

# Default values for the low-level graph engine
neostore.nodestore.db.mapped_memory=4G
#neostore.relationshipstore.db.mapped_memory=50M
#neostore.relationshipgroupstore.db.mapped_memory=10M
#neostore.propertystore.db.mapped_memory=90M
#neostore.propertystore.db.strings.mapped_memory=130M
#neostore.propertystore.db.arrays.mapped_memory=130M

CONF / Neo4j的-wrapper.conf：

wrapper.java.additional=-Dorg.neo4j.server.properties=conf/neo4j-server.properties
wrapper.java.additional=-Djava.util.logging.config.file=conf/logging.properties
wrapper.java.additional=-Dlog4j.configuration=file:conf/log4j.properties

#********************************************************************
# JVM Parameters
#********************************************************************

#wrapper.java.additional=-XX:+UseConcMarkSweepGC
wrapper.java.additional=-XX:+CMSClassUnloadingEnabled
wrapper.java.additional=-XX:-OmitStackTraceInFastThrow

# Own GC
wrapper.java.additional=-XX:+UseG1GC

# Uncomment the following lines to enable garbage collection logging
wrapper.java.additional=-Xloggc:data/log/neo4j-gc.log
wrapper.java.additional=-XX:+PrintGCDetails
wrapper.java.additional=-XX:+PrintGCDateStamps
wrapper.java.additional=-XX:+PrintGCApplicationStoppedTime
wrapper.java.additional=-XX:+PrintPromotionFailure
wrapper.java.additional=-XX:+PrintTenuringDistribution


# Java Heap Size: by default the Java heap size is dynamically
# calculated based on available system resources.
# Uncomment these lines to set specific initial and maximum
# heap size in MB.
wrapper.java.initmemory=8192
wrapper.java.maxmemory=8192

#********************************************************************
# Wrapper settings
#********************************************************************
# path is relative to the bin dir
wrapper.pidfile=../data/neo4j-server.pid

#********************************************************************
# Wrapper Windows NT/2000/XP Service Properties
#********************************************************************
# WARNING - Do not modify any of these properties when an application
#  using this configuration file has been installed as a service.
#  Please uninstall the service before modifying this section.  The
#  service can then be reinstalled.

# Name of the service
wrapper.name=neo4j

# User account to be used for linux installs. Will default to current
# user if not se                                                       1,1

Neo4j：慢输入和使用什么垃圾收集器？

0 个答案: