如何将较大的csv文件加载到Neo4j中

时间:2018-06-21 18:47:42

标签: neo4j cypher

我正在尝试将一个较大的csv文件( 1458644行)加载到neo4j中,但是我仍然遇到此错误:

Neo.TransientError.General.OutOfMemoryError: There is not enough memory to perform the current task. Please try increasing 'dbms.memory.heap.max_size' in the neo4j configuration (normally in 'conf/neo4j.conf' or, if you you are using Neo4j Desktop, found through the user interface) or if you are running an embedded installation increase the heap by using '-Xmx' command line flag, and then restart the database.

即使我将dbms.memory.heap.max_size=1024m更改为m = megbite,也会再次发生相同的错误!

注意:的csv大小为 195.888 KB

这是我的代码:

load csv with headers from "file:///train.csv" as line
create(pl:pickup_location{latitude:toFloat(line.pickup_latitude),longitude:toFloat(line.pickup_longitude)}),(pt:pickup_time{pickup:line.pickup_datetime}),(dl:dropoff_location{latitude:toFloat(line.dropoff_latitude),longitude:toFloat(line.dropoff_longitude)}),(dt:dropoff_time{dropoff:line.dropoff_datetime})
create (pl)-[:TLR]->(pt),(dl)-[:TLR]->(dt),(pl)-[:Trip]->(dl);

我该怎么办?

2 个答案:

答案 0 :(得分:0)

您应使用periodic commits批量处理CSV数据。例如,这将一次处理10,000行(默认批处理大小为1000):

USING PERIOD COMMIT 10000
LOAD CSV WITH HEADERS FROM "file:///train.csv" as line
CREATE (pl:pickup_location{latitude:toFloat(line.pickup_latitude),longitude:toFloat(line.pickup_longitude)}),(pt:pickup_time{pickup:line.pickup_datetime}),(dl:dropoff_location{latitude:toFloat(line.dropoff_latitude),longitude:toFloat(line.dropoff_longitude)}),(dt:dropoff_time{dropoff:line.dropoff_datetime})
CREATE (pl)-[:TLR]->(pt),(dl)-[:TLR]->(dt),(pl)-[:Trip]->(dl);

答案 1 :(得分:0)

我通过复制有限行here

的解决方案来解决该问题

这是我的解决方案:

AbsolutePath

此解决方案的缺点是您需要知道大型csv文件的行数( excel 无法打开大型csv文件)。