我在OrientDB中插入大量(几百万)边缘时遇到问题。在java中,我有一个包含~5000行和列的矩阵,其中每个条目表示对象之间的相似性值。我想将此信息插入OrientDB。
但是在创建了200k到400k之间的边缘之后,发生了ODatabaseException,我还没有找到解决这个问题的方法。
我正在通过以下方式创建边缘:使用SQL Batch,我将SQL查询添加到字符串并为每100个边执行批处理。每个SQL行都如下所示:
CEATE EDGE similarity FROM #10:0 TO #10:1 SET score = 0.5
我应该以不同的方式做这件事吗?是缺乏记忆问题吗?我一直无法找到解决方案。
感谢。
修改:
我正在使用异常和一些代码更新问题:
代码:
public void saveHouseCharacteristicsSimilarities(double[][] houseSim,
ArrayList<OHouse> houseList) {
OrientGraphFactory factory = new OrientGraphFactory("plocal:/directory");
OrientGraphNoTx graph = factory.getNoTx();
int counter=0;
String query = "BEGIN\n";
for (int h1 = 0; h1 < (houseList.size() - 1); h1++) {
for (int h2 = (h1 + 1); h2 < houseList.size(); h2++) {
counter++;
query += "create edge similarity from "
+ houseList.get(h1).getRid() + " to "
+ houseList.get(h2).getRid() + " SET score = "
+ houseSim[h1][h2] + "\n";
if (counter % 100 == 0
|| ((h1 == (houseList.size() - 1)) && (h2 == (houseList
.size() - 1)))) {
query += "COMMIT";
graph.command(new OCommandScript("sql", query)).execute();
query = "BEGIN";
}
}
}
graph.shutdown();
}
异常(在此特定点我已经保存了250000条边):
com.orientechnologies.orient.core.exception.ODatabaseException: Error on retrieving record #24:1880 (cluster: house)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.executeReadRecord(ODatabaseDocumentTx.java:1605)
at com.orientechnologies.orient.core.tx.OTransactionOptimistic.loadRecord(OTransactionOptimistic.java:226)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.load(ODatabaseDocumentTx.java:1424)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.load(ODatabaseDocumentTx.java:117)
at com.orientechnologies.orient.core.id.ORecordId.getRecord(ORecordId.java:313)
at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.getVertex(OrientBaseGraph.java:731)
at com.orientechnologies.orient.graph.sql.OCommandExecutorSQLCreateEdge$1.call(OCommandExecutorSQLCreateEdge.java:144)
at com.orientechnologies.orient.graph.sql.OCommandExecutorSQLCreateEdge$1.call(OCommandExecutorSQLCreateEdge.java:126)
at com.orientechnologies.orient.graph.sql.OGraphCommandExecutorSQLFactory.runInTx(OGraphCommandExecutorSQLFactory.java:123)
at com.orientechnologies.orient.graph.sql.OGraphCommandExecutorSQLFactory.runInTx(OGraphCommandExecutorSQLFactory.java:141)
at com.orientechnologies.orient.graph.sql.OCommandExecutorSQLCreateEdge.execute(OCommandExecutorSQLCreateEdge.java:126)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:64)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:1184)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:1173)
at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:63)
at com.orientechnologies.orient.core.command.script.OCommandExecutorScript.executeSQLScript(OCommandExecutorScript.java:335)
at com.orientechnologies.orient.core.command.script.OCommandExecutorScript.executeSQL(OCommandExecutorScript.java:138)
at com.orientechnologies.orient.core.command.script.OCommandExecutorScript.executeInContext(OCommandExecutorScript.java:82)
at com.orientechnologies.orient.core.command.script.OCommandExecutorScript.execute(OCommandExecutorScript.java:73)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:1184)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:1173)
at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:63)
at com.tinkerpop.blueprints.impls.orient.OrientGraphCommand.execute(OrientGraphCommand.java:48)
at com.ubiwhere.livinx.recEngine.dao.OHouseDAO.saveHouseCharacteristicsSimilarities(OHouseDAO.java:465)
at com.ubiwhere.livinx.recEngine.structures.CharacteristicsSimilarity.buildCharacteristicsMatrix(CharacteristicsSimilarity.java:33)
at com.ubiwhere.livinx.rest.house.HouseFavouriteService.getFavourites(HouseFavouriteService.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:152)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:271)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:254)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1028)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:381)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:344)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:221)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1682)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:344)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:316)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:174)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:415)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:282)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:459)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:167)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:201)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:175)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:235)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
at org.glassfish.grizzly.ProcessorExecutor.execute(Proc
答案 0 :(得分:0)
尝试这样的东西来创建边
OrientVertex vertex = graph.getVertex(houseList.get(h1).getRid());
OrientVertex vertex2 = graph.getVertex(houseList.get(h2).getRid());
vertex.addEdge("similarity",vertex2,new Object[]{"score",houseSim[h1][h2]});
答案 1 :(得分:0)
好吧,问题似乎不是东方本身,而是程序运行的glassfish服务器。由于某种原因,Glassfish终止了与东方的联系。无论如何,在服务器外运行程序解决了这个问题。谢谢!