我使用的是neo4j 1.9.4嵌入式模型。执行短路查询(50万个节点和600万个关系)时花了太长时间。我阅读了手册并找到了两个属性:
1. conf/neo4j-server.properties
org.neo4j.server.webserver.limit.executiontime=1000
2. conf/neo4j.properties
execution_guard_enabled=true
但问题是,似乎没有' neo4j-server.properties'嵌入式模型中的文件。如何设置?或者“超时”'嵌入式模型不支持属性吗?
请帮忙。非常感谢!
代码:
PathFinder<Path> finder = GraphAlgoFactory.shortestPath(Traversal
.expanderForTypes(RelationshipTypes.r01, Direction.BOTH)
.add(RelationshipTypes.r02, Direction.BOTH), 10);
paths = finder.findAllPaths(node1, node2);
for (Path shortestPath : paths) {
List<NodeBean> nodeList = new ArrayList<NodeBean>();
List<String> rList = new ArrayList<String>();
... ...
}
message.log的一部分:
2014-11-24 02:06:42.107+0000 INFO [org.neo4j]: --- INITIALIZED diagnostics END ---
2014-11-24 02:06:42.217+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.relationshiptypestore.db.names non clean shutdown detected
2014-11-24 02:06:42.217+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\neostore.relationshiptypestore.db.names] brickCount=0 brickSize=0b mappedMem=0b (storeSize=114b)
2014-11-24 02:06:42.232+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.relationshiptypestore.db non clean shutdown detected
2014-11-24 02:06:42.232+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\neostore.relationshiptypestore.db] brickCount=0 brickSize=0b mappedMem=0b (storeSize=10b)
2014-11-24 02:06:42.232+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.propertystore.db.strings non clean shutdown detected
2014-11-24 02:06:42.232+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\neostore.propertystore.db.strings] brickCount=10 brickSize=20864b mappedMem=20971520b (storeSize=209664b)
2014-11-24 02:06:42.232+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.propertystore.db.index.keys non clean shutdown detected
2014-11-24 02:06:42.232+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\neostore.propertystore.db.index.keys] brickCount=50 brickSize=20938b mappedMem=20971520b (storeSize=1048572b)
2014-11-24 02:06:42.232+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.propertystore.db.index non clean shutdown detected
2014-11-24 02:06:42.232+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\neostore.propertystore.db.index] brickCount=261 brickSize=20970b mappedMem=20971520b (storeSize=5473170b)
2014-11-24 02:06:42.232+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.propertystore.db.arrays non clean shutdown detected
2014-11-24 02:06:42.232+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\neostore.propertystore.db.arrays] brickCount=0 brickSize=20864b mappedMem=20971520b (storeSize=128b)
2014-11-24 02:06:42.232+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.propertystore.db non clean shutdown detected
2014-11-24 02:06:42.248+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\neostore.propertystore.db] brickCount=23513 brickSize=104837b mappedMem=104857600b (storeSize=2465619255b)
2014-11-24 02:06:42.248+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.relationshipstore.db non clean shutdown detected
2014-11-24 02:06:42.248+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\neostore.relationshipstore.db] brickCount=100000 brickSize=221826b mappedMem=104857600b (storeSize=22183075860b)
2014-11-24 02:06:42.248+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.nodestore.db non clean shutdown detected
2014-11-24 02:06:42.248+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\neostore.nodestore.db] brickCount=4733 brickSize=104877b mappedMem=104857600b (storeSize=496395405b)
2014-11-24 02:06:42.248+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore non clean shutdown detected
2014-11-24 02:06:42.248+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\neostore] brickCount=0 brickSize=0b mappedMem=0b (storeSize=54b)
2014-11-24 02:06:42.263+0000 INFO [org.neo4j]: Non clean shutdown detected on log [D:\FTP\graph1.db\nioneo_logical.log.1]. Recovery started ...
2014-11-24 02:06:42.279+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\nioneo_logical.log.1] logVersion=4730 with committed tx=1072646
2014-11-24 02:06:42.279+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\nioneo_logical.log.1] entries found=0 lastEntryPos=16
2014-11-24 02:06:42.279+0000 INFO [org.neo4j]: Opened logical log [D:\FTP\graph1.db\nioneo_logical.log.1] version=4730, lastTxId=-1 (recovered)
2014-11-24 02:06:42.279+0000 INFO [org.neo4j]: XaResourceManager[nioneo_logical.log] sorting 0 xids
2014-11-24 02:06:42.279+0000 INFO [org.neo4j]: XaResourceManager[nioneo_logical.log] checkRecoveryComplete 0 xids
2014-11-24 02:06:42.419+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.relationshiptypestore.db.names rebuild id generator, highId=3 defragged count=0
2014-11-24 02:06:42.934+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.propertystore.db.index.keys rebuild id generator, highId=3 defragged count=0
2014-11-24 02:06:46.351+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.propertystore.db.index rebuild id generator, highId=2 defragged count=0
2014-11-24 02:06:46.444+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.propertystore.db.strings rebuild id generator, highId=1490 defragged count=0
2014-11-24 02:06:46.538+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.propertystore.db.arrays rebuild id generator, highId=1 defragged count=0
2014-11-24 02:06:46.694+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.propertystore.db rebuild id generator, highId=60134068 defragged count=0
2014-11-24 02:06:46.943+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.relationshipstore.db rebuild id generator, highId=672185416 defragged count=0
2014-11-24 02:06:47.255+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore.nodestore.db rebuild id generator, highId=55117690 defragged count=0
2014-11-24 02:06:47.349+0000 INFO [org.neo4j]: D:\FTP\graph1.db\neostore rebuild id generator, highId=6 defragged count=0
2014-11-24 02:06:47.411+0000 INFO [org.neo4j]: XaResourceManager[nioneo_logical.log] recovery completed.
2014-11-24 02:06:47.411+0000 INFO [org.neo4j]: Recovery on log [D:\FTP\graph1.db\nioneo_logical.log.1] completed.
2014-11-24 02:06:47.411+0000 INFO [org.neo4j]: TM opening log: D:\FTP\graph1.db\tm_tx_log.1
2014-11-24 02:06:47.567+0000 INFO [org.neo4j]: BackupServer communication server started and bound to /0.0.0.0:6362
2014-11-24 02:06:47.599+0000 INFO [org.neo4j]: Non clean shutdown detected on log [D:\FTP\graph1.db\index\lucene.log.1]. Recovery started ...
2014-11-24 02:06:47.599+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\index\lucene.log.1] logVersion=252 with committed tx=5514
2014-11-24 02:06:47.599+0000 INFO [org.neo4j]: [D:\FTP\graph1.db\index\lucene.log.1] entries found=0 lastEntryPos=16
2014-11-24 02:06:47.599+0000 INFO [org.neo4j]: Opened logical log [D:\FTP\graph1.db\index\lucene.log.1] version=252, lastTxId=-1 (recovered)
2014-11-24 02:06:47.599+0000 INFO [org.neo4j]: XaResourceManager[lucene.log] sorting 0 xids
2014-11-24 02:06:47.599+0000 INFO [org.neo4j]: XaResourceManager[lucene.log] checkRecoveryComplete 0 xids
2014-11-24 02:06:47.599+0000 INFO [org.neo4j]: XaResourceManager[lucene.log] recovery completed.
2014-11-24 02:06:47.599+0000 INFO [org.neo4j]: Recovery on log [D:\FTP\graph1.db\index\lucene.log.1] completed.
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: --- STARTED diagnostics for KernelDiagnostics:Versions START ---
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: Graph Database: org.neo4j.kernel.EmbeddedGraphDatabase StoreId[time:1414568675229, id:-3029933565465970314, store version: 13561656364791302]
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: Kernel version: Neo4j - Graph Database Kernel 1.9.4
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: Neo4j component versions:
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: Neo4j - Graph Database Kernel 1.9.4
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: --- STARTED diagnostics for KernelDiagnostics:Versions END ---
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: --- STARTED diagnostics for NEO_STORE_VERSIONS START ---
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: Store versions:
Store versions:
NeoStore v0.A.0
NodeStore v0.A.0
RelationshipStore v0.A.0
RelationshipTypeStore v0.A.0
PropertyStore v0.A.0
PropertyIndexStore v0.A.0
StringPropertyStore v0.A.0
ArrayPropertyStore v0.A.0
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: --- STARTED diagnostics for NEO_STORE_VERSIONS END ---
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: --- STARTED diagnostics for NEO_STORE_ID_USAGE START ---
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: Id usage:
Id usage:
NodeStore: used=55155045 high=55155044
RelationshipStore: used=672214420 high=672214419
RelationshipTypeStore: used=2 high=1
PropertyStore: used=60137055 high=60137054
PropertyIndexStore: used=608130 high=608129
StringPropertyStore: used=1638 high=1637
ArrayPropertyStore: used=1 high=0
2014-11-24 02:06:47.614+0000 INFO [org
.neo4j]: --- STARTED diagnostics for NEO_STORE_ID_USAGE END ---
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: --- STARTED diagnostics for PERSISTENCE_WINDOW_POOL_STATS START ---
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: --- STARTED diagnostics for PERSISTENCE_WINDOW_POOL_STATS END ---
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: --- STARTED diagnostics for KernelDiagnostics:StoreFiles START ---
2014-11-24 02:06:47.614+0000 INFO [org.neo4j]: Disk space on partition (Total / Free / Free %): 316763721728 / 32676884480 / 10
Storage files: (filename : modification date - size)
active_tx_log: 2014-11-21T08:52:51+0800 - 11.00 B
index:
lucene:
node:
nodes:
segments.gen: 2014-11-21T12:28:14+0800 - 20.00 B
segments_4z: 2014-11-21T12:28:14+0800 - 5.25 kB
_111.fdt: 2014-11-21T12:28:53+0800 - 223.40 MB
_111.fdx: 2014-11-21T12:28:24+0800 - 55.69 MB
_111.fnm: 2014-11-21T12:28:24+0800 - 18.00 B
_111.frq: 2014-11-21T12:28:34+0800 - 53.68 MB
_111.nrm: 2014-11-21T12:28:35+0800 - 6.96 MB
_111.prx: 2014-11-21T12:28:37+0800 - 13.92 MB
_111.tii: 2014-11-21T12:28:38+0800 - 1.62 MB
_111.tis: 2014-11-21T12:28:58+0800 - 115.02 MB
_1ri.fdt: 2014-11-21T12:29:40+0800 - 263.73 MB
_1ri.fdx: 2014-11-21T12:29:09+0800 - 63.93 MB
_1ri.fnm: 2014-11-21T12:29:10+0800 - 18.00 B
_1ri.frq: 2014-11-21T12:29:21+0800 - 61.92 MB
_1ri.nrm: 2014-11-21T12:29:22+0800 - 7.99 MB
_1ri.prx: 2014-11-21T12:29:25+0800 - 15.98 MB
_1ri.tii: 2014-11-21T12:29:25+0800 - 1.84 MB
_1ri.tis: 2014-11-21T12:29:48+0800 - 129.59 MB
_2lv.fdt: 2014-11-21T12:30:33+0800 - 303.36 MB
_2lv.fdx: 2014-11-21T12:30:01+0800 - 73.55 MB
_2lv.fnm: 2014-11-21T12:30:01+0800 - 18.00 B
_2lv.frq: 2014-11-21T12:30:14+0800 - 71.53 MB
_2lv.nrm: 2014-11-21T12:30:16+0800 - 9.19 MB
_2lv.prx: 2014-11-21T12:30:19+0800 - 18.39 MB
_2lv.tii: 2014-11-21T12:30:20+0800 - 2.14 MB
_2lv.tis: 2014-11-21T12:30:46+0800 - 150.79 MB
_3gr.fdt: 2014-11-21T12:31:28+0800 - 311.85 MB
_3gr.fdx: 2014-11-21T12:31:00+0800 - 75.61 MB
_3gr.fnm: 2014-11-21T12:31:00+0800 - 18.00 B
_3gr.frq: 2014-11-21T12:31:13+0800 - 73.59 MB
_3gr.nrm: 2014-11-21T12:31:15+0800 - 9.45 MB
_3gr.prx: 2014-11-21T12:31:18+0800 - 18.90 MB
_3gr.tii: 2014-11-21T12:31:19+0800 - 2.18 MB
_3gr.tis: 2014-11-21T12:31:46+0800 - 154.25 MB
_4ah.fdt: 2014-11-21T12:32:23+0800 - 306.20 MB
_4ah.fdx: 2014-11-21T12:31:59+0800 - 74.23 MB
_4ah.fnm: 2014-11-21T12:31:59+0800 - 18.00 B
_4ah.frq: 2014-11-21T12:32:12+0800 - 72.22 MB
_4ah.nrm: 2014-11-21T12:32:14+0800 - 9.28 MB
_4ah.prx: 2014-11-21T12:32:17+0800 - 18.56 MB
_4ah.tii: 2014-11-21T12:32:17+0800 - 2.14 MB
_4ah.tis: 2014-11-21T12:32:44+0800 - 152.60 MB
_4d9.cfs: 2014-11-21T12:32:34+0800 - 65.42 MB
_4g1.cfs: 2014-11-21T12:32:46+0800 - 65.54 MB
_4hz.cfs: 2014-11-21T12:32:46+0800 - 6.42 MB
_4ij.cfs: 2014-11-21T12:32:47+0800 - 6.44 MB
_4it.cfs: 2014-11-21T12:32:57+0800 - 65.35 MB
_4ll.cfs: 2014-11-21T12:32:58+0800 - 65.73 MB
_4od.cfs: 2014-11-21T12:33:09+0800 - 65.89 MB
_4on.cfs: 2014-11-21T12:33:00+0800 - 6.50 MB
_4ox.cfs: 2014-11-21T12:33:01+0800 - 6.57 MB
_4p7.cfs: 2014-11-21T12:33:02+0800 - 6.57 MB
_4ph.cfs: 2014-11-21T12:33:03+0800 - 6.60 MB
_4pr.cfs: 2014-11-21T12:33:05+0800 - 6.60 MB
_4px.cfs: 2014-11-21T12:33:05+0800 - 718.54 kB
_4py.cfs: 2014-11-21T12:33:05+0800 - 723.03 kB
_4pz.cfs: 2014-11-21T12:33:05+0800 - 719.12 kB
_4q1.cfs: 2014-11-21T12:33:07+0800 - 6.59 MB
_gf.fdt: 2014-11-21T12:33:29+0800 - 131.96 MB
_gf.fdx: 2014-11-21T12:33:15+0800 - 33.04 MB
_gf.fnm: 2014-11-21T12:33:15+0800 - 18.00 B
_gf.frq: 2014-11-21T12:33:20+0800 - 31.02 MB
_gf.nrm: 2014-11-21T12:33:21+0800 - 4.13 MB
_gf.prx: 2014-11-21T12:33:23+0800 - 8.26 MB
_gf.tii: 2014-11-21T12:33:23+0800 - 1.01 MB
_gf.tis: 2014-11-21T12:33:33+0800 - 73.32 MB
- Total: 2014-11-24T10:06:47+0800 - 3.51 GB
- Total: 2014-11-21T12:28:14+0800 - 3.51 GB
- Total: 2014-11-21T12:28:14+0800 - 3.51 GB
lucene-store.db: 2014-11-22T17:39:30+0800 - 40.00 B
lucene.log.1: 2014-11-21T18:34:32+0800 - 16.00 B
lucene.log.active: 2014-11-21T18:33:31+0800 - 4.00 B
- Total: 2014-11-22T11:40:56+0800 - 3.51 GB
index.db: 2014-11-21T08:52:48+0800 - 103.00 B
lock: 2014-11-22T15:32:04+0800 - 0.00 B
messages.log: 2014-11-24T10:06:44+0800 - 37.60 MB
neostore: 2014-11-24T10:06:42+0800 - 54.00 B
neostore.id: 2014-11-24T10:06:47+0800 - 9.00 B
neostore.nodestore.db: 2014-11-22T15:32:04+0800 - 473.40 MB
neostore.nodestore.db.id: 2014-11-24T10:06:47+0800 - 9.00 B
neostore.propertystore.db: 2014-11-22T15:32:04+0800 - 2.30 GB
neostore.propertystore.db.arrays: 2014-11-22T15:32:04+0800 - 128.00 B
neostore.propertystore.db.arrays.id: 2014-11-24T10:06:46+0800 - 9.00 B
neostore.propertystore.db.id: 2014-11-24T10:06:46+0800 - 9.00 B
neostore.propertystore.db.index: 2014-11-22T17:03:23+0800 - 5.22 MB
neostore.propertystore.db.index.id: 2014-11-24T10:06:46+0800 - 9.00 B
neostore.propertystore.db.index.keys: 2014-11-22T15:32:04+0800 - 1024.00 kB
neostore.propertystore.db.index.keys.id: 2014-11-24T10:06:42+0800 - 9.00 B
neostore.propertystore.db.strings: 2014-11-22T15:32:04+0800 - 204.75 kB
neostore.propertystore.db.strings.id: 2014-11-24T10:06:46+0800 - 9.00 B
neostore.relationshipstore.db: 2014-11-22T15:32:04+0800 - 20.66 GB
neostore.relationshipstore.db.id: 2014-11-24T10:06:46+0800 - 9.00 B
neostore.relationshiptypestore.db: 2014-11-24T10:06:42+0800 - 10.00 B
neostore.relationshiptypestore.db.id: 2014-11-24T10:06:42+0800 - 9.00 B
neostore.relationshiptypestore.db.names: 2014-11-22T15:32:04+0800 - 114.00 B
neostore.relationshiptypestore.db.names.id: 2014-11-24T10:06:42+0800 - 9.00 B
nioneo_logical.log.1: 2014-11-22T15:32:23+0800 - 16.00 B
nioneo_logical.log.active: 2014-11-22T15:32:04+0800 - 4.00 B
nioneo_logical.log.v4729: 2014-11-22T15:31:23+0800 - 16.00 B
store_lock: 2014-11-21T12:21:37+0800 - 0.00 B
tm_tx_log.1: 2014-11-24T10:06:47+0800 - 0.00 B
tm_tx_log.2: 2014-11-21T12:21:38+0800 - 28.68 kB
2014-11-24 02:06:47.630+0000 INFO [org.neo4j]: --- STARTED diagnostics for KernelDiagnostics:StoreFiles END ---
2014-11-24 02:06:47.630+0000 INFO [org.neo4j]: Started - database is now available
2014-11-24 02:06:47.630+0000 INFO [org.neo4j]: GC Monitor started.
2014-11-24 02:24:00.340+0000 INFO [org.neo4j]: GC Monitor: Application threads blocked for an additional 142ms [total block time: 0.142s]
2014-11-24 02:42:44.771+0000 INFO [org.neo4j]: GC Monitor: Application threads blocked for an additional 373ms [total block time: 0.515s]
neo4j.properties:
dump_configuration=false
use_memory_mapped_buffers:true
neostore.nodestore.db.mapped_memory=100M
neostore.relationshipstore.db.mapped_memory=100M
neostore.propertystore.db.mapped_memory=100M
neostore.propertystore.db.strings.mapped_memory=20M
neostore.propertystore.db.arrays.mapped_memory=20M /* file cache*/
neostore.propertystore.db.index.keys.mapped_memory=20M
neostore.propertystore.db.index.mapped_memory=20M
cache_type=soft
cache.memory_radio=50%
node_cache_size=2000M
relationship_cache_size=800M
excution_guard_enabled=true
keep_logical_logs=false
online_backup_enabled=true
ha.pull_interval=10
答案 0 :(得分:0)
Direction.BOTH表示它是圆圈。
您的MMIO设置应该更大(更接近文件大小), 你的堆也可能(因为它包括Windows上的mmio部分 - &gt;至少12G到16G堆)。
您没有共享messages.log中有关堆大小的指示。
neostore.nodestore.db.mapped_memory=500M
neostore.relationshipstore.db.mapped_memory=6G
neostore.propertystore.db.mapped_memory=1G
neostore.propertystore.db.strings.mapped_memory=1G
如果您仍然运行企业,则应使用:
cache_type=hpc
或
cache_type=none
对于Guard,除了设置你已经拥有的配置外,添加它的代码是这样的:
((GraphDatabaseAPI)graphDB).getDependencyResolver().resolveDependency(Guard.class);
guard.startTimeout(timeLimit);
guard.startOperationsCount(maxOps);
try {
return doOperation();
} catch (GuardException e) {
LOG.warn("Aborting Operation ");
throw new RuntimeException(e);
} finally {
guard.stop();
}