经过反复试验,我设法将数据强制为一组CSV文件,这些文件最初由Neo4j-admin导入接受。但是,在最大文件导入的大约70%的情况下,程序因Unsorted data, a > b Failure
错误而崩溃。我还没有发现需要对数据进行排序的任何信息,而且我不确定在这种情况下应该对哪些列进行排序。我清除了所有NAN值的数据,以防万一这是问题所在,但尚未解决。
如果有人有任何想法,请在下面查看完整日志。谢谢!
bash-3.2$ bin/neo4j-admin import --id-type INTEGER --nodes:User "import/user_nodes.csv" --nodes:Step "import/step_nodes.csv" --relationships:IS_ROOT_OF "import/root_edges.csv" --relationships:COLLECTED_AT_STEP "import/step_edges.csv" --max-memory=8g
Neo4j version: 3.5.8
Importing the contents of these files into /Users/James/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-fc317459-d0c3-43c0-b1eb-d16df6b0d172/installation-3.5.8/data/databases/graph.db:
Nodes:
:User
/Users/James/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-fc317459-d0c3-43c0-b1eb-d16df6b0d172/installation-3.5.8/import/user_nodes.csv
:Step
/Users/James/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-fc317459-d0c3-43c0-b1eb-d16df6b0d172/installation-3.5.8/import/step_nodes.csv
Relationships:
:IS_ROOT_OF
/Users/James/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-fc317459-d0c3-43c0-b1eb-d16df6b0d172/installation-3.5.8/import/root_edges.csv
:COLLECTED_AT_STEP
/Users/James/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-fc317459-d0c3-43c0-b1eb-d16df6b0d172/installation-3.5.8/import/step_edges.csv
Available resources:
Total machine memory: 16.00 GB
Free machine memory: 3.45 GB
Max heap memory : 4.44 GB
Processors: 8
Configured max memory: 10.40 GB
High-IO: true
Import starting 2019-09-17 13:24:26.679+0100
Estimated number of nodes: 11.60 M
Estimated number of node properties: 92.81 M
Estimated number of relationships: 25.60 M
Estimated number of relationship properties: 0.00
Estimated disk space usage: 2.27 GB
Estimated required memory usage: 1.14 GB
InteractiveReporterInteractions command list (end with ENTER):
c: Print more detailed information about current stage
i: Print more detailed information
(1/4) Node import 2019-09-17 13:24:26.905+0100
Estimated number of nodes: 11.60 M
Estimated disk space usage: 1.46 GB
Estimated required memory usage: 1.14 GB
.......... .......... .......... .......... .......... 5% ∆3s 933ms
.......... .......... .......... .......... .......... 10% ∆2s 816ms
.......... .......... .......... .......... .......... 15% ∆2s 412ms
.......... .......... .......... .......... .......... 20% ∆3s 31ms
.......... .......... .......... .......... .......... 25% ∆2s 227ms
.......... .......... .......... .......... .......... 30% ∆2s 624ms
.......... ....-..... .......... .......... .......... 35% ∆754ms
.......... .......... .......... .......... .......... 40% ∆1ms
.......... .......... .......... .......... .......... 45% ∆0ms
.......... .......... .......... .......... .......... 50% ∆1s 406ms
.......... .......... .......... .......... .......... 55% ∆1s 5ms
.......... .......... .......... .......... .......... 60% ∆2s 822ms
.......... .......... .......... .......... .......... 65% ∆3s 215ms
.......... .......... .......... .......... .........Exception in thread "DETECT-6" java.lang.RuntimeException: java.lang.IllegalStateException: Unsorted data, a > b Failure:[10526719] 2efffffbfcd78001 > 2cc08cb655d4d000 | 4194302:3154733
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers$Worker.run(Workers.java:131)
Caused by: java.lang.IllegalStateException: Unsorted data, a > b Failure:[10526719] 2efffffbfcd78001 > 2cc08cb655d4d000 | 4194302:3154733
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.detect(EncodingIdMapper.java:367)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.run(EncodingIdMapper.java:335)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers$Worker.run(Workers.java:126)
Exception in thread "DETECT-5" java.lang.RuntimeException: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers$Worker.run(Workers.java:131)
Caused by: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.detect(EncodingIdMapper.java:367)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.run(EncodingIdMapper.java:335)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers$Worker.run(Workers.java:126)
Exception in thread "Thread-5" java.lang.RuntimeException: java.lang.RuntimeException: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
at org.neo4j.unsafe.impl.batchimport.staging.AbstractStep.issuePanic(AbstractStep.java:155)
at org.neo4j.unsafe.impl.batchimport.staging.AbstractStep.issuePanic(AbstractStep.java:147)
at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:59)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers.awaitAndThrowOnError(Workers.java:82)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers.awaitAndThrowOnErrorStrict(Workers.java:90)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.detectAndMarkCollisions(EncodingIdMapper.java:442)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.prepare(EncodingIdMapper.java:236)
at org.neo4j.unsafe.impl.batchimport.IdMapperPreparationStep.process(IdMapperPreparationStep.java:56)
at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:53)
... 1 more
Caused by: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.detect(EncodingIdMapper.java:367)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.run(EncodingIdMapper.java:335)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers$Worker.run(Workers.java:126)
. 70% ∆1s 791ms
.......... .......... .......... .......... .......... 75% ∆0ms
.......... .......... .......... .......... .......... 80% ∆1ms
.......... .......... .......... .......... .......... 85% ∆0ms
.......... .......... .......... .......... .......... 90% ∆0ms
.......... .......... .......... .......... .......... 95% ∆0ms
.......... .......... .......... .......... .......... 100% ∆1ms
IMPORT FAILED in 29s 410ms.
Data statistics is not available.
Peak memory usage: 1.09 GB
Import error: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
Caused by:java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
java.lang.RuntimeException: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers.awaitAndThrowOnError(Workers.java:82)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers.awaitAndThrowOnErrorStrict(Workers.java:90)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.detectAndMarkCollisions(EncodingIdMapper.java:442)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.prepare(EncodingIdMapper.java:236)
at org.neo4j.unsafe.impl.batchimport.IdMapperPreparationStep.process(IdMapperPreparationStep.java:56)
at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:53)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.detect(EncodingIdMapper.java:367)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.run(EncodingIdMapper.java:335)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers$Worker.run(Workers.java:126)
WARNING Import failed. The store files in /Users/James/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-fc317459-d0c3-43c0-b1eb-d16df6b0d172/installation-3.5.8/data/databases/graph.db are left as they are, although they are likely in an unusable state. Starting a database on these store files will likely fail or observe inconsistent records so start at your own risk or delete the store manually
unexpected error: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
bash-3.2$
答案 0 :(得分:0)
在预感到我的ID字段有问题后,我将我的原始ID字段复制到新列中以保存(如果我需要将记录与原始源匹配),并创建了一个新的ID列简单的整数范围。
然后解决了问题,但是发生了一个新问题,我认为这可以解释原始问题。我新创建的原始ID字段引发错误,因为这些数字太长而无法整数。我将此列从整数类型重新分配为Long类型,此问题已解决。所以我的理论是,用于ID的原始值太长而无法存储为整数。