我最近在Windows Server 2012 R2上将Neo4j从版本2.1.7升级到2.2.0。我一直在运行我的数据收集脚本,使用.cql文本文件中的以下cypher命令加载到数据库中:
USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'file:X:\\parsed\\TOI\\0031\\node_full_statuses.csv' AS line MERGE (ts:TwitterStatus {id: toInt(line.id)}) ON CREATE SET ts.id_str = line.id_str, ts.timestamp = toint(line.timestamp), ts.created_at = line.created_at, ts.favorites_count = toInt(line.favorite_count), ts.lang = line.lang, ts.source = line.source, ts.retweet_count = toInt(line.retweet_count), ts.text = line.text, ts.truncated = line.truncated ON MATCH SET ts.id_str = line.id_str, ts.timestamp = toint(line.timestamp), ts.last_updated = timestamp(), ts.created_at = line.created_at, ts.favorites_count = toInt(line.favorite_count), ts.lang = line.lang, ts.source = line.source, ts.retweet_count = toInt(line.retweet_count), ts.text = line.text, ts.truncated = line.truncated;
这在2.1.7中没有任何错误。现在当我使用相同的命令时:
PS C:\neo4j-enterprise-2.2.0\bin> .\Neo4jShell.bat -file X:\parsed\NDTV\0031\load_database.cql -v
我得到以下异常:
ERROR (-v for expanded information): TransactionFailureException: Transaction was marked as successful, but unable to commit transaction so rolled back. org.neo4j.graphdb.TransactionFailureException: Transaction was marked as successful, but unable to commit transaction so rolled back. at org.neo4j.kernel.TopLevelTransaction.close(TopLevelTransaction.java:126) at org.neo4j.cypher.internal.spi.v2_2.TransactionBoundQueryContext.commitAndRestartTx(TransactionBoundQueryContext.scala:304) at org.neo4j.cypher.internal.compiler.v2_2.spi.DelegatingQueryContext.commitAndRestartTx(DelegatingQueryContext.scala:99) at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext.org$neo4j$cypher$internal$compatibility$ExceptionTranslatingQueryContext$$super$commitAndRestartTx(ExceptionTranslatingQueryContext.scala:134) at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext$$anonfun$commitAndRestartTx$1.apply$mcV$sp(ExceptionTranslatingQueryContext.scala:134) at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext$$anonfun$commitAndRestartTx$1.apply(ExceptionTranslatingQueryContext.scala:134) at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext$$anonfun$commitAndRestartTx$1.apply(ExceptionTranslatingQueryContext.scala:134) at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext.org$neo4j$cypher$internal$compatibility$ExceptionTranslatingQueryContext$$translateException(ExceptionTranslatingQueryContext.scala:182) at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext.commitAndRestartTx(ExceptionTranslatingQueryContext.scala:134) at org.neo4j.cypher.internal.compiler.v2_2.spi.DelegatingQueryContext.commitAndRestartTx(DelegatingQueryContext.scala:99) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.LoadCsvPeriodicCommitObserver.org$neo4j$cypher$internal$compiler$v2_2$executionplan$LoadCsvPeriodicCommitObserver$$commitAndRestartTx(LoadCsvPeriodicCommitObserver.scala:50) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.LoadCsvPeriodicCommitOb$$$$19575187a71575c7a3df5a5d253113c$$$$ommitObserver$$onNext$1.apply$mcV$sp(LoadCsvPeriodicCommitObserver.scala:46) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.UpdateCounter.resetIfPastLimit(UpdateCounter.scala:34) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.LoadCsvPeriodicCommitObserver.org$neo4j$cypher$internal$compiler$v2_2$executionplan$LoadCsvPeriodicCommitObserver$$onNext(LoadCsvPeriodicCommitObserver.scala:46) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.LoadCsvPeriodicCommitObserver$$anonfun$1.apply$mcV$sp(LoadCsvPeriodicCommitObserver.scala:36) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.LoadCsvIterator.next(LoadCsvIterator.scala:31) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.LoadCsvIterator.next(LoadCsvIterator.scala:24) at scala.collection.Iterator$$anon$11.next(Iterator.scala:328) at scala.collection.Iterator$$anon$13.next(Iterator.scala:372) at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371) at org.neo4j.cypher.internal.compiler.v2_2.pipes.EmptyResultPipe.internalCreateResults(EmptyResultPipe.scala:29) at org.neo4j.cypher.internal.compiler.v2_2.pipes.PipeWithSource.createResults(Pipe.scala:118) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.DefaultExecutionResultBuilderFactory$ExecutionWorkflowBuilder.createResults(DefaultExecutionResultBuilderFactory.scala:92) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.DefaultExecutionResultBuilderFactory$ExecutionWorkflowBuilder.build(DefaultExecutionResultBuilderFactory.scala:64) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1.apply(ExecutionPlanBuilder.scala:122) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1.apply(ExecutionPlanBuilder.scala:106) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.ExecutionPlanBuilder$$anon$1.run(ExecutionPlanBuilder.scala:70) at org.neo4j.cypher.internal.compatibility.CompatibilityFor2_2$ExecutionPlanWrapper$$anonfun$run$1.apply(CompatibilityFor2_2.scala:166) at org.neo4j.cypher.internal.compatibility.CompatibilityFor2_2$ExecutionPlanWrapper$$anonfun$run$1.apply(CompatibilityFor2_2.scala:166) at org.neo4j.cypher.internal.compatibility.exceptionHandlerFor2_2$.runSafely(CompatibilityFor2_2.scala:104) at org.neo4j.cypher.internal.compatibility.CompatibilityFor2_2$ExecutionPlanWrapper.run(CompatibilityFor2_2.scala:165) at org.neo4j.cypher.internal.PreparedPlanExecution.execute(PreparedPlanExecution.scala:27) at org.neo4j.cypher.ExecutionEngine.execute(ExecutionEngine.scala:107) at org.neo4j.cypher.ExecutionEngine.execute(ExecutionEngine.scala:101) at org.neo4j.cypher.javacompat.internal.ServerExecutionEngine.executeQuery(ServerExecutionEngine.java:61) at org.neo4j.shell.kernel.apps.cypher.Start.getResult(Start.java:86) at org.neo4j.shell.kernel.apps.cypher.Using.getResult(Using.java:53) at org.neo4j.shell.kernel.apps.cypher.Start.exec(Start.java:68) at org.neo4j.shell.kernel.apps.TransactionProvidingApp.execute(TransactionProvidingApp.java:223) at org.neo4j.shell.impl.AbstractAppServer.interpretLine(AbstractAppServer.java:117) at org.neo4j.shell.kernel.GraphDatabaseShellServer.interpretLine(GraphDatabaseShellServer.java:93) at org.neo4j.shell.impl.RemotelyAvailableServer.interpretLine(RemotelyAvailableServer.java:60) at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source) at sun.rmi.transport.Transport$1.run(Unknown Source) at sun.rmi.transport.Transport$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Unknown Source) at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$254(Unknown Source) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$2/1427081262.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: org.neo4j.kernel.impl.store.MultipleUnderlyingStorageExceptions: Errors when closing (flushing) index updaters: (:label[5](property[0])) PreexistingIndexEntryConflictException{propertyValue=581543886377459712, addedNodeId=58110492, existingNodeId=57966629} at org.neo4j.kernel.impl.api.index.IndexUpdaterMap.close(IndexUpdaterMap.java:101) at org.neo4j.kernel.impl.api.index.IndexingService$3.close(IndexingService.java:527) at org.neo4j.kernel.impl.api.TransactionRepresentationCommitProcess.commit(TransactionRepresentationCommitProcess.java:86) at org.neo4j.kernel.impl.api.KernelTransactionImplementation.commit(KernelTransactionImplementation.java:501) at org.neo4j.kernel.impl.api.KernelTransactionImplementation.close(KernelTransactionImplementation.java:417) at org.neo4j.kernel.TopLevelTransaction.close(TopLevelTransaction.java:112) ... 58 more Suppressed: org.neo4j.kernel.impl.store.UnderlyingStorageException: PreexistingIndexEntryConflictException{propertyValue=581543886377459712, addedNodeId=58110492, existingNodeId=57966629} at org.neo4j.kernel.impl.api.index.IndexUpdaterMap.close(IndexUpdaterMap.java:93) ... 63 more Caused by: PreexistingIndexEntryConflictException{propertyValue=581543886377459712, addedNodeId=58110492, existingNodeId=57966629} at org.neo4j.kernel.impl.api.index.UniquePropertyIndexUpdater.close(UniquePropertyIndexUpdater.java:102) at org.neo4j.kernel.impl.api.index.UpdateCountingIndexUpdater.close(UpdateCountingIndexUpdater.java:65) at org.neo4j.kernel.impl.api.index.ContractCheckingIndexProxy$1.close(ContractCheckingIndexProxy.java:104) at org.neo4j.kernel.impl.api.index.IndexUpdaterMap.close(IndexUpdaterMap.java:85) ... 63 more -host Domain name or IP of host to connect to (default: localhost) -port Port of host to connect to (default: 1337) -name RMI name, i.e. rmi://<host>:<port>/<name> (default: shell) -pid Process ID to connect to -c Command line to execute. After executing it the shell exits -file File containing commands to execute, or '-' to read from stdin. After executing it the shell exits -readonly Connect in readonly mode (only for connecting with -path) -path Points to a neo4j db path so that a local server can be started there -config Points to a config file when starting a local server Example arguments for remote: -port 1337 -host 192.168.1.234 -port 1337 -name shell -host localhost -readonly ...or no arguments for default values Example arguments for local: -path /path/to/db -path /path/to/db -config /path/to/neo4j.config -path /path/to/db -readonly
我觉得这很奇怪,因为我在使用merge命令之前没有从duplicates中获得任何错误(id和id_str是唯一约束)。这似乎表明它是导致回滚的PreexistingIndexEntryConflictException。
如果我再次运行命令行命令,我会得到一个不同的例外:
PS C:\neo4j-enterprise-2.2.0\bin> .\Neo4jShell.bat -file X:\parsed\NDTV\0031\load_database.cql -v ERROR (-v for expanded information): QueryExecutionKernelException: Node 58109560 already exists with label TwitterStatus and property "id_str"=[581520425747759104] org.neo4j.kernel.impl.query.QueryExecutionKernelException: Node 58109560 already exists with label TwitterStatus and property "id_str"=[581520425747759104] at org.neo4j.cypher.javacompat.internal.ServerExecutionEngine.executeQuery(ServerExecutionEngine.java:65) at org.neo4j.shell.kernel.apps.cypher.Start.getResult(Start.java:86) at org.neo4j.shell.kernel.apps.cypher.Using.getResult(Using.java:53) at org.neo4j.shell.kernel.apps.cypher.Start.exec(Start.java:68) at org.neo4j.shell.kernel.apps.TransactionProvidingApp.execute(TransactionProvidingApp.java:223) at org.neo4j.shell.impl.AbstractAppServer.interpretLine(AbstractAppServer.java:117) at org.neo4j.shell.kernel.GraphDatabaseShellServer.interpretLine(GraphDatabaseShellServer.java:93) at org.neo4j.shell.impl.RemotelyAvailableServer.interpretLine(RemotelyAvailableServer.java:60) at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source) at sun.rmi.transport.Transport$1.run(Unknown Source) at sun.rmi.transport.Transport$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Unknown Source) at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$254(Unknown Source) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$2/1427081262.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: org.neo4j.cypher.CypherExecutionException: Node 58109560 already exists with label TwitterStatus and property "id_str"=[581520425747759104] at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext.org$neo4j$cypher$internal$compatibility$ExceptionTranslatingQueryContext$$translateException(ExceptionTranslatingQueryContext.scala:184) at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext$ExceptionTranslatingOperations.setProperty(ExceptionTranslatingQueryContext.scala:151) at org.neo4j.cypher.internal.compiler.v2_2.spi.UpdateCountingQueryContext$CountingOps.setProperty(UpdateCountingQueryContext.scala:132) at org.neo4j.cypher.internal.compiler.v2_2.mutation.PropertySetAction.exec(PropertySetAction.scala:59) at org.neo4j.cypher.internal.compiler.v2_2.mutation.MergeNodeAction$$anonfun$exec$1.apply(MergeNodeAction.scala:80) at org.neo4j.cypher.internal.compiler.v2_2.mutation.MergeNodeAction$$anonfun$exec$1.apply(MergeNodeAction.scala:80) at scala.collection.immutable.List.foreach(List.scala:318) at org.neo4j.cypher.internal.compiler.v2_2.mutation.MergeNodeAction.exec(MergeNodeAction.scala:79) at org.neo4j.cypher.internal.compiler.v2_2.pipes.ExecuteUpdateCommandsPipe.org$neo4j$cypher$internal$compiler$v2_2$pipes$ExecuteUpdateCommandsPipe$$exec(ExecuteUpdateCommandsPipe.scala:56) at org.neo4j.cypher.internal.compiler.v2_2.pipes.ExecuteUpdateCommandsPi$$$$3fd0ddfab96df657454927b510aaae$$$$ands$1$$anonfun$apply$2.apply(ExecuteUpdateCommandsPipe.scala:45) at org.neo4j.cypher.internal.compiler.v2_2.pipes.ExecuteUpdateCommandsPi$$$$3fd0ddfab96df657454927b510aaae$$$$ands$1$$anonfun$apply$2.apply(ExecuteUpdateCommandsPipe.scala:45) at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371) at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371) at org.neo4j.cypher.internal.compiler.v2_2.pipes.EmptyResultPipe.internalCreateResults(EmptyResultPipe.scala:29) at org.neo4j.cypher.internal.compiler.v2_2.pipes.PipeWithSource.createResults(Pipe.scala:118) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.DefaultExecutionResultBuilderFactory$ExecutionWorkflowBuilder.createResults(DefaultExecutionResultBuilderFactory.scala:92) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.DefaultExecutionResultBuilderFactory$ExecutionWorkflowBuilder.build(DefaultExecutionResultBuilderFactory.scala:64) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1.apply(ExecutionPlanBuilder.scala:122) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1.apply(ExecutionPlanBuilder.scala:106) at org.neo4j.cypher.internal.compiler.v2_2.executionplan.ExecutionPlanBuilder$$anon$1.run(ExecutionPlanBuilder.scala:70) at org.neo4j.cypher.internal.compatibility.CompatibilityFor2_2$ExecutionPlanWrapper$$anonfun$run$1.apply(CompatibilityFor2_2.scala:166) at org.neo4j.cypher.internal.compatibility.CompatibilityFor2_2$ExecutionPlanWrapper$$anonfun$run$1.apply(CompatibilityFor2_2.scala:166) at org.neo4j.cypher.internal.compatibility.exceptionHandlerFor2_2$.runSafely(CompatibilityFor2_2.scala:104) at org.neo4j.cypher.internal.compatibility.CompatibilityFor2_2$ExecutionPlanWrapper.run(CompatibilityFor2_2.scala:165) at org.neo4j.cypher.internal.PreparedPlanExecution.execute(PreparedPlanExecution.scala:27) at org.neo4j.cypher.ExecutionEngine.execute(ExecutionEngine.scala:107) at org.neo4j.cypher.ExecutionEngine.execute(ExecutionEngine.scala:101) at org.neo4j.cypher.javacompat.internal.ServerExecutionEngine.executeQuery(ServerExecutionEngine.java:61) ... 24 more Caused by: org.neo4j.kernel.api.exceptions.schema.UniqueConstraintViolationKernelException: Node 58109560 already exists with label 5 and property 2=581520425747759104 at org.neo4j.kernel.impl.api.ConstraintEnforcingEntityOperations.validateNoExistingNodeWithLabelAndProperty(ConstraintEnforcingEntityOperations.java:119) at org.neo4j.kernel.impl.api.ConstraintEnforcingEntityOperations.nodeSetProperty(ConstraintEnforcingEntityOperations.java:97) at org.neo4j.kernel.impl.api.LockingStatementOperations.nodeSetProperty(LockingStatementOperations.java:324) at org.neo4j.kernel.impl.api.OperationsFacade.nodeSetProperty(OperationsFacade.java:657) at org.neo4j.cypher.internal.spi.v2_2.TransactionBoundQueryContext$NodeOperations.setProperty(TransactionBoundQueryContext.scala:176) at org.neo4j.cypher.internal.compiler.v2_2.spi.DelegatingOperations.setProperty(DelegatingQueryContext.scala:126) at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext$ExceptionTranslatingOperations.org$neo4j$cypher$internal$compatibility$ExceptionTranslatingQuery Context$ExceptionTranslatingOperations$$super$setProperty(ExceptionTranslatingQueryContext.scala:151) at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext$ExceptionTranslatingOperations$$anonfun$setProperty$1.apply$mcV$sp(ExceptionTranslatingQueryContext.scala:151) at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext$ExceptionTranslatingOperations$$anonfun$setProperty$1.apply(ExceptionTranslatingQueryContext.scala:151) at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext$ExceptionTranslatingOperations$$anonfun$setProperty$1.apply(ExceptionTranslatingQueryContext.scala:151) at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContext.org$neo4j$cypher$internal$compatibility$ExceptionTranslatingQueryContext$$translateException(ExceptionTranslatingQueryContext.scala:182) ... 51 more -host Domain name or IP of host to connect to (default: localhost) -port Port of host to connect to (default: 1337) -name RMI name, i.e. rmi://<host>:<port>/<name> (default: shell) -pid Process ID to connect to -c Command line to execute. After executing it the shell exits -file File containing commands to execute, or '-' to read from stdin. After executing it the shell exits -readonly Connect in readonly mode (only for connecting with -path) -path Points to a neo4j db path so that a local server can be started there -config Points to a config file when starting a local server Example arguments for remote: -port 1337 -host 192.168.1.234 -port 1337 -name shell -host localhost -readonly ...or no arguments for default values Example arguments for local: -path /path/to/db -path /path/to/db -config /path/to/neo4j.config -path /path/to/db -readonly
这是1000行提交周期开始时的事务。所有1000行都在数据库中。 (我已经单独删除它们并获得相同的异常,并且nodeID增加1)我需要删除初始错误事务所在的提交周期的1000行以使脚本继续前进。
我的问题是:有没有办法解决这个问题,以便合并正常运行,而不会在Neo4j说看起来不错的事务上获得这个奇怪的异常,但它不会提交?
我不认为我的工作非常好,并且不喜欢任何建议。