我尝试导入372M节点,它给了我以下异常。异常提出后,程序无法停止并正常运行。但它可能会运行几个小时而不会对磁盘使用情况,CPU和内存进行任何更改。好像被冻结了,只是“排序”的东西。我已经替换了一些特殊字符。怎么解决?
以下是我的剧本:
#!/bin/bash
cd /cygdrive/d/neo4j-enterprise-2.2.0/bin
users="d:\\RiskNetwork\\nodes_header.txt"
for i in /cygdrive/d/RiskNetwork/Nodes/*
do
users=$users,$(cygpath -w "$i")
done
./Neo4jImport.bat --stacktrace --into ../data/weibo.db --nodes:User $users --delimiter ^A --array-delimiter ^B --quote ^C --id-type STRING
这是输出和例外:
......
......
[*>:18.42 MB/s-----------------|PROPERTIES(2)==========|NODE:2.72 GB------------
[*>:18.42 MB/s-----------------|PROPERTIES(2)==========|NODE:2.73 GB------------
[*>:18.41 MB/s------------------|PROPERTIES(2)========|NODE:2.73 GB-------------
[*>:18.40 MB/s-----------------|PROPERTIES(2)=========|NODE:2.74 GB-------------
[*>:18.40 MB/s----------------|PROPERTIES(2)==========|NODE:2.75 GB-------------
[*>:18.38 MB/s-----------------|PROPERTIES(2)==========|NODE:2.76 GB-----------|
[*>:18.37 MB/s-------------------|PROPERTIES(2)=========|NODE:2.76 GB-----------
[*>:18.36 MB/s---------------|PROPERTIES(2)===========|NODE:2.77 GB-------------
[*>:18.36 MB/s------------------|PROPERTIES(2)==========|NODE:2.78 GB------------|v:57.94 MB/s(]372M
Done in 17m 14s 300ms
Prepare node index
[*SPLIT:2.85 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
[*SPLIT:4.11 GB-----------------------------------------------------------------
......
......
[*SPLIT:4.16 GB-------------------------------------------------------------------
[*SPLIT:4.16 GB-------------------------------------------------------------------
[*SPLIT:4.16 GB-------------------------------------------------------------------
[*SPLIT:4.16 GB-------------------------------------------------------------------
[*SPLIT:4.16 GB-------------------------------------------------------------------
[*SPLIT:4.16 GB-------------------------------------------------------------------
[*SPLIT:4.16 GB-------------------------------------------------------------------
[*SPLIT:4.17 GB-------------------------------------------------------------------
[*SPLIT:4.17 GB-------------------------------------------------------------------
[*SPLIT:4.17 GB-------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB---------------------------------------------------------------------------------]
0Exception in thread "SortWorker-0" java.lang.ArrayIndexOutOfBoundsException: Requested index -1, but length is 1000000
at org.neo4j.unsafe.impl.batchimport.cache.OffHeapNumberArray.addressOf(OffHeapNumberArray.java:55)
at org.neo4j.unsafe.impl.batchimport.cache.OffHeapLongArray.get(OffHeapLongArray.java:42)
at org.neo4j.unsafe.impl.batchimport.cache.DynamicLongArray.get(DynamicLongArray.java:42)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.partition(ParallelSort.java:204)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:246)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.access$100(ParallelSort.java:39)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort$SortWorker.run(ParallelSort.java:296)
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
[*SORT:4.17 GB--------------------------------------------------------------------
......
......
这些例外的含义是什么?我可以肯定的是,分隔符,数组分隔符和引号不会出现在输入文件的数据列上下文中。
答案 0 :(得分:1)
https://github.com/neo4j/neo4j/pull/4427将解决此问题,因此希望Neo4j版本2.2.1将包含此修复程序。
答案 1 :(得分:1)
我有同样的错误(但使用SortWorker-1而不是0)并删除“空白”字段解决了它,正如评论中运气所示。
“空白字段”,我的意思是空格/标签/ ...&他们的组合。