通过许多可选关系优化Cypher查询

时间:2014-05-18 10:43:45

标签: rest neo4j cypher

我在使用Neo4j 2.0.1的批处理REST API上使用Cypher。

我正在尝试优化具有大量可选关系的查询。我想一次性检索所有数据,以限制我必须对数据库进行的往返次数。尽管我的数据库中只有大约12000个节点,但查询已经开始爬行(有些节点需要1.5秒才能返回1000个节点)。

我已经在http://gist.neo4j.org/?9494429e3cbbbeda2b11设置了一个更详细的图表要点。

我的查询通常采用以下形式:

MATCH (u:user { id: "u1" })
WITH u

MATCH u-[:CONTACT]->(c:contact)
WITH u, c

OPTIONAL MATCH  (c)-[:CREATED]->(xca:activity)<-[:USERACTIVITY]-(xcc:contact)
OPTIONAL MATCH  (c)-[:HISTORY]->(xcu:activity)<-[:USERACTIVITY]-(xuc:contact)
OPTIONAL MATCH (c)-[:PHONE]->(xp:phone)
OPTIONAL MATCH (c)-[:ADDRESS]->(xa:address)
OPTIONAL MATCH (u)-[:PHONE]->(xup:phone)
OPTIONAL MATCH (u)-[:ADDRESS]->(xua:address)
WITH DISTINCT c AS x, u,
    COLLECT(DISTINCT xp) AS xps,
    COLLECT(DISTINCT xa) AS xas,
    COLLECT(DISTINCT xup) AS xups,
    COLLECT(DISTINCT xua) AS xuas,
    xca.createdat AS createdat,
    xcu.createdat AS updatedat,
    {id: xcc.id} AS createdby,
    {id: xuc.id} AS updatedby
RETURN COLLECT({
    id: x.id,
    name:  COALESCE(u.name, x.name), 
    createdat: createdat,
    createdby: createdby,
    updatedat: updatedat,
    updatedby: updatedby,
        phones: (CASE WHEN size(xps)= 0
          THEN NULL
          ELSE [xp IN xps | { id: xp.id, number: xp.number}]
          END),
        userphones: (CASE WHEN size(xups)= 0
          THEN NULL
          ELSE [xup IN xups | { id: xup.id, number: xup.number }]
          END),
        addresses: (CASE WHEN size(xas)= 0
          THEN NULL
          ELSE [xa IN xas | { id: xa.id, city: xa.city}]
          END),
        useraddresses: (CASE WHEN size(xuas)= 0
          THEN NULL
          ELSE [xua IN xuas | { id: xua.id, city: xua.city}]
          END)
}) AS r

有没有更好的方法来查询具有大量可选关系的节点?在这样的情况下,我应该期待什么类型的表现?谢谢!

更新

我根据Michael的建议重写了查询,并验证每一步的基数为1(也就是说,查询的每一步只返回一行)。当我只返回一个联系人时,查询大约需要400毫秒(比原始查询差大约5倍)。

然而,当我尝试针对返回1000个联系人的较大数据集运行查询时,它现在只是挂起,永远不会完成,我必须重新启动Neo4j服务器。我没有尝试将结果合并到一张新地图中,但我认为这不会解决问题。我是否创建了一个更差的交叉产品,现在我在逐步测试查询时没有出现?

  MATCH (u:user { id: "123" })
  WITH u

  MATCH (u)-[:CONTACT]->(c:contact)
  WITH c

  OPTIONAL MATCH
    (c)-[:CREATED]->(xca:activity)-[:USERACTIVITY*1..4]<-(xcc:contact),
    (c)-[:HISTORY]->(xcu:activity)-[:USERACTIVITY*1..4]<-(xuc:contact)
  WITH c AS x,
    xca.createdat AS createdat, xcu.createdat AS updatedat,
    {id: xcc.id, object: xcc.object} AS createdby,
    {id: xuc.id, object: xuc.object} AS updatedby

  OPTIONAL MATCH
    (x)-[:PHONE]->(xp:phone)
  WITH x, createdat, updatedat, createdby, updatedby,
    COLLECT(xp) as xps

  OPTIONAL MATCH  
    (x)-[:ADDRESS]->(xa:address)
  WITH x, createdat, updatedat, createdby, updatedby, xps,
    COLLECT(xa) as xas

  OPTIONAL MATCH (xu:user)-[:CONTACT]->(x)
  OPTIONAL MATCH (xu)-[:PHONE]->(xup:phone)
  WITH x, createdat, updatedat, createdby, updatedby, xps, xas,
    xu, COLLECT(xup) as xups

  OPTIONAL MATCH (xu)-[:ADDRESS]->(xua:address)
  WITH x, createdat, updatedat, createdby, updatedby, xps, xas,
    xu, xups, COLLECT(xua) as xuas

  RETURN COLLECT({
    id: x.id,
    object: x.object,
    status: x.status,
    teamid: x.teamid,
    name:  COALESCE(xu.name, x.name),
    displayname: COALESCE(xu.displayname, x.displayname),
    email: COALESCE(xu.email, x.email),
    imageurl: COALESCE(xu.imageurl, x.imageurl),
    workhours: x.workhours,
    notes: x.notes,
    company: x.company,
    createdat: createdat,
    createdby: createdby,
    updatedat: updatedat,
    updatedby: updatedby,
    isuser: (NOT xu IS NULL),
    phones: (CASE WHEN size(xps)= 0
      THEN NULL
      ELSE [xp IN xps | { id: xp.id, object: xp.object,
            number: xp.number, description: xp.description }]
      END),
    userphones: (CASE WHEN size(xups)= 0
      THEN NULL
      ELSE [xup IN xups | { id: xup.id, object: xup.object,
            number: xup.number, description: xup.description }]
      END),
    addresses: (CASE WHEN size(xas)= 0
      THEN NULL
      ELSE [xa IN xas | { id: xa.id, object: xa.object,
            street: xa.street, locality: xa.locality, region: xa.region,
            postcode: xa.postcode, country: xa.country, description: xa.description, neighborhood: xa.neighborhood }]
      END),
    useraddresses: (CASE WHEN size(xuas)= 0
      THEN NULL
      ELSE [xua IN xuas | { id: xua.id, object: xua.object,
            street: xua.street, locality: xua.locality, region: xua.region,
            postcode: xua.postcode, country: xua.country, description: xua.description, neighborhood: xua.neighborhood }]
      END)
  }) AS r

更新2

我尝试移动手机和地址,但它没有效果,我甚至拿出它们仍然看到类似的结果(1000个联系人超过2s)。我已经消除了查询中的所有复杂性,只是为了查看基线是什么。执行以下查询平均需要385ms:

MATCH (t:team {id:"123"})
WITH t

MATCH (c:contact)-[:CONTACT]->(t)
WITH c AS x

RETURN COLLECT({
    id: x.id,
    object: x.object,
    status: x.status,
    teamid: x.teamid,
    name:  x.name,
    displayname: x.displayname,
    email: x.email,
    imageurl: x.imageurl,
    workhours: x.workhours,
    notes: x.notes,
    company: x.company
}) AS r

我的数据库有6000个节点和12000个关系,此查询返回1000个联系人(整个数据库大小为7 MB)。预计这类查询差不多400毫秒?

我真的很感谢看到我的数据库,但我想我真的想知道如何自己诊断这些问题。当我使用Web UI时,我看不到爆炸(每个结果只返回1行)。当我使用PROFILE命令时,我看不到你想要的数百万的数字。

是否有其他工具可用于诊断性能问题?是否有某种调试器可以追踪问题?

1 个答案:

答案 0 :(得分:0)

问题是您在所有比赛之间创建了交叉产品。

如果您可以识别最多只有一个连接的匹配项,则可以预先提取它们。否则,您可以收集匹配的信息,以恢复您的基数1(或联系人#m)。

e.g。

MATCH (u:user { id: "u1" })
OPTIONAL MATCH (u)-[:PHONE]->(xup:phone)
OPTIONAL MATCH (u)-[:ADDRESS]->(xua:address)
// cardinality 1
WITH u, collect(distinct xup) as phones, collect(distinct xua) as addresses
MATCH (u)-[:CONTACT]->(c:contact)
WITH u, c, phones, addresses
OPTIONAL MATCH (c)-[:CREATED]->(xca:activity)<-[:USERACTIVITY]-(xcc:contact)
WITH u,c, phones,addresses, collect(distinct xcc) as contact_activities
...

您已经使用了地图文字,因此您也可以将它们与我建议的方法相结合,通过逐步向地图(或集合)添加键

e.g。

MATCH (u:user { id: "u1" })
OPTIONAL MATCH (u)-[:PHONE]->(xup:phone)
OPTIONAL MATCH (u)-[:ADDRESS]->(xua:address)
// cardinality 1
WITH u, {user:u, phones:collect(distinct xup), addresses: collect(distinct xua)} as user_info
MATCH (u)-[:CONTACT]->(c:contact)
WITH c, user_info
OPTIONAL MATCH (c)-[:CREATED]->(xca:activity)<-[:USERACTIVITY]-(xcc:contact)
WITH c, user_info, {activities: collect(distinct xcc)} as contact_info
...

DISTINCT和聚合也很可能无法帮助你。聚合已经为分组键创建了不同的条目。

我尝试调整你的图形主义者(感谢提供btw)来展示它的样子(但我没有完全通过):http://gist.neo4j.org/?bba019835045ed352925

您可能对此图形主义者感兴趣:A complex query result projection in cypher