对于大量数据,neo4j真的需要大的RAM大小吗?

时间:2015-02-25 23:41:25

标签: performance neo4j

我们现有的数据是节点存储有近65MB的数据和260MB的关系存储?

在进入制作阶段时,预计会有200名用户同时使用该应用程序,这会在幕后触发neo4j查询。我通过了neo4j计算器,建议使用64G RAM。

目前我测试的分段有3个RHEL服务器,每个服务器有8G内存,并且已经为java堆分配了4G。它有HDD而不是SSD。每个服务器2个cpu核心。

作为单个用户,简单查询带来了快速,而一些复杂的查询需要在20秒内运行更大的数据节点。它一直很好,直到2到3个用户。但是在一段时间之后,结果不会返回,并且查询会不确定地运行。

我不确定这类数据的预期系统配置是什么,管理层希望证明对更高系统采购的要求是合理的。

有没有机会,可以调整此配置以支持生产负载。

我已经按照neo4j网站的建议进行了性能调整,从linux性能到内存缓存性能参数。

或者是否需要有更大的RAM来将所有关系加载到内存中,以使查询和并发加载具有更快的响应


查询问题

match (c:company) 
where c.company IN [ "GENERAL ELECTRIC-TELEPRESENCE","PNOC_REG02","testzenos2","testzenos3","PT10","CMSP_SLT_SYNC","CMSP_SLT_SYNC_2","Smoke Test Company","PTrans-Regr","SPWIFI-POC","PNOC_REG01","IBM","MERAKISP-MERAKICUST","0101rms","Meraki_SP01-new3" ] 

match (c)-[r1]->(s:physical_location)-[r5]->(im:im_tkt) 
where ( 
   ( im.assigned_group_id IN [ "SGP000000000259" ] and im.company = "GENERAL ELECTRIC-TELEPRESENCE") 
OR ( im.assigned_group_id IN [ "SGP000000000259" ] and im.company = "PNOC_REG02") 
OR ( im.assigned_group_id IN [ "SGP000000000259","SGP000000000175" ] and im.company = "testzenos2") 
OR ( im.assigned_group_id IN [ "SGP000000000259","SGP000000000175" ] and im.company = "testzenos3") 
OR ( im.assigned_group_id IN [ "SGP000000000259","SGP000000000088" ] and im.company = "PT10") 
OR ( im.assigned_group_id IN [ "SGP000000000175","SGP000000000088" ] and im.company = "CMSP_SLT_SYNC") 
OR ( im.assigned_group_id IN [ "SGP000000000175","SGP000000000088" ] and im.company = "CMSP_SLT_SYNC_2") 
OR ( im.assigned_group_id IN [ "SGP000000000088" ] and im.company = "Smoke Test Company") 
OR ( im.assigned_group_id IN [ "SGP000000000088" ] and im.company = "PTrans-Regr") 
OR ( im.assigned_group_id IN [ "SGP000000000088" ] and im.company = "SPWIFI-POC") 
OR ( im.assigned_group_id IN [ "SGP000000000088" ] and im.company = "PNOC_REG01") 
OR ( im.assigned_group_id IN [ "SGP000000000088" ] and im.company = "IBM") 
OR ( im.assigned_group_id IN [ "SGP000000000088" ] and im.company = "MERAKISP-MERAKICUST") 
OR ( im.assigned_group_id IN [ "SGP000000000088" ] and im.company = "0101rms") 
OR ( im.assigned_group_id IN [ "SGP000000000088" ] and im.company = "Meraki_SP01-new3") ) 

RETURN collect(distinct(im.incident_number)) as im_tkt__incident_number , 
       collect(distinct(im.company)) as im_tkt__company , 
       collect(distinct(im.assigned_support_organization)) as im_tkt__assigned_support_organization , 
       collect(distinct(im.assignee)) as im_tkt__assignee , 
       collect(distinct(im.description)) as im_tkt__description , 
       collect(distinct(im.priority)) as im_tkt__priority , 
       collect(distinct(im.status)) as im_tkt__status , 
       collect(distinct(im.impact)) as im_tkt__impact , 
       collect(distinct(im.entryid_TS1)) as im_tkt__entryid_TS1;

这是一个运行时间较长的查询


1 个答案:

答案 0 :(得分:0)

您是否创建了索引或约束?

create index on :company(company);
create index on :im_tkt(assigned_group_id);

在真实程序中使用参数!!

传递param的地图列表,如下所示:

[{company:"GENERAL ELECTRIC-TELEPRESENCE", 
  assigned_group_ids:[ "SGP000000000259" ]}, 
...]

然后查询:

UNWIND {param} as pair
MATCH (c:company {company:pair.company})-[r1]->(s:physical_location)-[r5]->(im:im_tkt)
WHERE im.assigned_group_id IN pair.assigned_group_ids
RETURN collect(distinct(im.incident_number)) as im_tkt__incident_number , 
       collect(distinct(im.company)) as im_tkt__company , 
       collect(distinct(im.assigned_support_organization)) as im_tkt__assigned_support_organization , 
       collect(distinct(im.assignee)) as im_tkt__assignee , 
       collect(distinct(im.description)) as im_tkt__description , 
       collect(distinct(im.priority)) as im_tkt__priority , 
       collect(distinct(im.status)) as im_tkt__status , 
       collect(distinct(im.impact)) as im_tkt__impact , 
       collect(distinct(im.entryid_TS1)) as im_tkt__entryid_TS1;

进行测试可以使用

UNWIND [{company:"GENERAL ELECTRIC-TELEPRESENCE", assigned_group_ids:[ "SGP000000000259" ]}] as pair
MATCH (c:company {company:pair.company})-[r1]->(s:physical_location)-[r5]->(im:im_tkt)
WHERE im.assigned_group_id IN pair.assigned_group_ids
RETURN ...