为什么Neo4j APOC Cypher导出会忽略unwindBatchSize命令?

时间:2019-06-15 19:28:06

标签: neo4j cypher neo4j-apoc

我正在使用Neo4j(3.5.1)apoc-3.5.0.4插件导出Neo4j DB的子图。

在Cypher导出中,我总是有一大块UNWIND节点(> 500)。

据我对https://neo4j-contrib.github.io/neo4j-apoc-procedures/3.5/export-import/#export-cypher上的文档的了解,批量大小“部分”的默认限制为100。我的导出创建了更大的UNWIND块。

当尝试通过Cypher Shell导入导出的Cypher文件时,导入恰好在UNWIND的第一个大块处停止,并出现以下错误:

无效的输入'{':预期的标识符字符,空格,FROM GRAPH,CONSTRUCT,LOAD CSV,START,MATCH,UNWIND,MERGE,CREATE UNIQUE,CREATE,SET,DELETE,REMOVE,FOREACH,WITH,CALL,RETURN ,UNION,';'或输入结尾(第2行,第2593列,偏移量:2593))

提到的文档说批量大小可以由

控制
useOptimizations:{unwindBatchSize: 10}

用于限制UNWIND条目数量的参数。

但是当我将其设置为较小的值(例如10)以强制导出限制条目的UNWIND大小时,它可以在导出中的随机点上正常工作。在随机数量的行之后没有任何可识别的原因,导出开始构建大的捆绑节点块(最多1000个节点)。

我使用以下命令从Neo4j DB导出子图:

MATCH (a:LABEL1)-[r:LINK]->(b:LABEL1)
WHERE a.title IN [xyz,...]
WITH collect(a)+collect(b) as export_nodes, collect(r) as export_rels
CALL apoc.export.cypher.data(export_nodes,export_rels,"export.cypher",
{format:'plain',cypherFormat:'updateAll', useOptimizations:
{type:'unwind_batch',unwindBatchSize: 10}}) 
YIELD file, source, format, nodes, relationships, properties, time
RETURN nodes, relationships, time;

我正在使用以下命令导入Cypher文件:

$NEO4J_HOME/bin/cypher-shell -u myuser -p mypassword < 'export.cypher'

导入正常,直到出现错误。正确导入了Cypher文件中的所有节点,直到发生错误为止。

我稍微玩了一下unwindBatchSize命令,以检查它-至少在一定程度上是否正常。所以我也检查了5个节点的UNWIND限制。当突然出现一个巨大的块时,这会创建结构良好的5个块,直到200..500行。

有人提示如何纠正此行为吗?

非常感谢

克里德


更新

在为生成的Cypher输出创建匿名摘录时,我认识到可能不是unwindBatchSize是问题的主要原因。仔细查看数据,我发现换行符和缺少的Cypher命令MERGE似乎是导致错误的原因。

当查看索引位置9:605和9:1142(下面提供的导出内容的索引位置)时,您会发现应该有一个换行符和MERGE命令条目。取而代之的是,输出将继续显示内容。

我没有更改输出的任何结构(只是将标签,属性和内容匿名化了。)

CREATE CONSTRAINT ON (node:Entity) ASSERT (node.title) IS UNIQUE;
UNWIND [{title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}},  {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:1, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}] as row
MERGE (n:Entity{title: row.title}) SET n += row.properties;
UNWIND [{title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}] as row
MERGE (n:Entity{title: row.title}) SET n += row.properties;
...
<for the next 200 blocks everything works as expacted... when suddenly the MERGE line is no longer generated>
...
UNWIND [{title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:1, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:1, prop2:0, prop3:0}}] as row{title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:1, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:1, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}] as row{title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0, ...
...
<This behaviour repeats for a while an suddenly it falls back to the expacted blocks set by the unwindBatchSize>
...
CREATE CONSTRAINT ON (node:Entity) ASSERT (node.title) IS UNIQUE;
UNWIND [{title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:1, prop2:0, prop3:0}}, {title:"Xyz_Xyz", properties:{prop1:0, prop2:0, prop3:0}}] as row
MERGE (n:Entity{title: row.title}) SET n += row.properties;
...
<The exported structure for adding the relations appear to be ok>
...

0 个答案:

没有答案