不支持Ontotext参数defaut.min.distinct.threshold

时间：2019-08-08 12:47:51

标签： graphdb

Ontotext数据库抛出 NotEnoughMemoryForDistinctGroupBy: Insufficient free Heap Memory ... , threshold:250Mb 尽管参数defaut.min.distinct.threshold=1074000000 设置。

Ontotext引擎似乎没有考虑设置参数，因此我们仍然遇到错误。

我们正在Docker中使用Ontotext 8.10.1-Free。 运行容器，我们将为Xmx参数设置的容器提供尽可能多的内存

我们尝试了不同的内存设置Xms，Xmx和上述阈值参数。但是在日志中，它指出正在使用阈值的默认值250m。

这是我们的过程信息在运行时的外观：

>  ps auxw|grep java
root         1 85.4 47.1 5729308 964184 ?      Ssl  09:24   0:46 /docker-java-home/bin/java -XX:MaxDirectMemorySize=128G -Xms2g -Xmx2g -Ddefaut.min.distinct.threshold=1074000000  
-Djava.net.preferIPv4Stack=true -XX:+UseParallelGC -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/graphdb/dist/heapdump.hprof -XX:OnOutOfMemoryError=kill -9 %p
-Djava.awt.headless=true -Dfile.encoding=UTF-8 -Dgraphdb.foreground=yes -Dgraphdb.dist=/opt/graphdb/dist -cp /opt/graphdb/dist/lib/* -Dgraphdb.home=/opt/graphdb/home com.ontotext.graphdb.server.GraphDBWorkbench

但是，当我们查看 logs （主日志.... log）时，我们会看到类似这样的语句（请注意在开头添加两颗星的行）：

[INFO ] 2019-08-08 09:33:22,793 [repositories/kg-test-graph-import | c.o.c.AbstractParameter] Configured parameter 'FSIPC' to default value '2'
[INFO ] 2019-08-08 09:33:23,138 [repositories/kg-test-graph-import | c.o.g.s.StatementsController] POST SPARQL update request to repository
[INFO ] 2019-08-08 09:33:23,244 [repositories/kg-test-graph-import | c.o.g.s.StatementsController] POST SPARQL update request to repository
[INFO ] 2019-08-08 09:33:23,299 [repositories/kg-test-graph-import | c.o.f.s.RepositoryController] GET query -1149340244
[INFO ] 2019-08-08 09:33:23,334 [repositories/kg-test-graph-import | c.o.c.AbstractParameter] Configured parameter 'order.construct' to default value 'false'
**[INFO ] 2019-08-08 09:33:23,345 [repositories/kg-test-graph-import | c.o.c.AbstractParameter] Configured parameter 'default.min.distinct.threshold' to default value '250.0M'
[INFO ] 2019-08-08 09:33:23,357 [repositories/kg-test-graph-import | c.o.g.s.GraphQueryResultView] Request for query -1149340244 is finished
[INFO ] 2019-08-08 09:33:23,367 [repositories/kg-test-graph-import | c.o.g.s.StatementsController] POST SPARQL update request to repository
[INFO ] 2019-08-08 09:33:23,415 [repositories/kg-test-graph-import | c.o.g.s.StatementsController] POST SPARQL update request to repository
[INFO ] 2019-08-08 09:33:23,487 [repositories/kg-test-graph-import | c.o.g.s.StatementsController] POST SPARQL update request to repository
[INFO ] 2019-08-08 09:33:23,704 [repositories/kg-test-graph-import | c.o.f.s.RepositoryController] GET query 1759975767

此外，本体文本日志显示了一些例外，这些例外也可以在针对数据库运行我们的测试和Sparql请求的客户端中见证：

[WARN ] 2019-08-08 09:41:10,442 [repositories/kg-test-graph-import | c.o.t.u.DistinctMemoryMonitor] Insufficient free Heap Memory 238Mb for group by and distinct, threshold 250Mb,  reached 0Mb
[ERROR] 2019-08-08 09:41:10,469 [repositories/kg-test-graph-import | c.o.t.SailConnectionImpl] Memory limit for GroupBy/Distinct reached
com.ontotext.trree.util.NotEnoughMemoryForDistinctGroupBy: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
        at com.ontotext.trree.util.DistinctMemoryMonitor.updateGroupByMemory(DistinctMemoryMonitor.java:73)
        at com.ontotext.trree.util.LongKeyBase.<init>(LongKeyBase.java:38)
        at com.ontotext.trree.util.LongKeySet.<init>(LongKeySet.java:26)
        at com.ontotext.trree.query.OwlimDistinctIteration.<init>(OwlimDistinctIteration.java:20)
        at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:1086)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:639)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:228)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.TupleFunctionEvaluationStrategy.evaluate(TupleFunctionEvaluationStrategy.java:65)
        at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:135)
        at org.eclipse.rdf4j.query.algebra.evaluation.iterator.JoinIterator.<init>(JoinIterator.java:42)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:824)
        at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:237)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:796)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:230)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.TupleFunctionEvaluationStrategy.evaluate(TupleFunctionEvaluationStrategy.java:65)
        at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:135)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:650)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:228)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.TupleFunctionEvaluationStrategy.evaluate(TupleFunctionEvaluationStrategy.java:65)
        at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:135)
        at com.ontotext.trree.SailConnectionImpl.evaluateWithRepoConnectionInternal(SailConnectionImpl.java:1786)
        at com.ontotext.trree.SailConnectionImpl.evaluateWithRepoConnection(SailConnectionImpl.java:1679)
        at com.ontotext.trree.SailConnectionImpl.evaluateInternal(SailConnectionImpl.java:249)
        at com.ontotext.trree.SailConnectionImpl.evaluate(SailConnectionImpl.java:2316)
        at org.eclipse.rdf4j.repository.sail.helpers.SailUpdateExecutor.evaluateWhereClause(SailUpdateExecutor.java:487)
        at org.eclipse.rdf4j.repository.sail.helpers.SailUpdateExecutor.executeModify(SailUpdateExecutor.java:450)
        at org.eclipse.rdf4j.repository.sail.helpers.SailUpdateExecutor.executeUpdate(SailUpdateExecutor.java:121)
        at org.eclipse.rdf4j.repository.sail.SailUpdate.execute(SailUpdate.java:64)
        at com.ontotext.trree.monitorRepository.MonitorRepositoryConnection$1.execute(MonitorRepositoryConnection.java:382)
        at org.eclipse.rdf4j.http.server.repository.statements.StatementsController.getSparqlUpdateResult(StatementsController.java:254)
        at org.eclipse.rdf4j.http.server.repository.statements.StatementsController.handleRequestInternal(StatementsController.java:114)
        at com.ontotext.graphdb.sesame.StatementsController.handleRequestInternal(StatementsController.java:41)
        at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:177)
        at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:52)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:908)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at com.ontotext.forest.core.request.RequestFilter.doFilterInternal(RequestFilter.java:45)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at com.github.ziplet.filter.compression.CompressingFilter.doFilter(CompressingFilter.java:263)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at com.ontotext.forest.security.AdminDelegatingFilterProxy.doFilter(AdminDelegatingFilterProxy.java:38)
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357)
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1417)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
[ERROR] 2019-08-08 09:41:10,549 [repositories/kg-test-graph-import | c.o.f.s.GraphDBProtocolExceptionResolver] Error while handling request (500)
org.eclipse.rdf4j.http.server.ServerHTTPException: org.eclipse.rdf4j.query.UpdateExecutionException: org.eclipse.rdf4j.sail.SailException: com.ontotext.trree.util.NotEnoughMemoryForDistinctGroupBy: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
        at org.eclipse.rdf4j.http.server.repository.statements.StatementsController.getSparqlUpdateResult(StatementsController.java:266)
        at org.eclipse.rdf4j.http.server.repository.statements.StatementsController.handleRequestInternal(StatementsController.java:114)
        at com.ontotext.graphdb.sesame.StatementsController.handleRequestInternal(StatementsController.java:41)
        at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:177)
        at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:52)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:908)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at com.ontotext.forest.core.request.RequestFilter.doFilterInternal(RequestFilter.java:45)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at com.github.ziplet.filter.compression.CompressingFilter.doFilter(CompressingFilter.java:263)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at com.ontotext.forest.security.AdminDelegatingFilterProxy.doFilter(AdminDelegatingFilterProxy.java:38)
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357)
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1417)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.eclipse.rdf4j.query.UpdateExecutionException: org.eclipse.rdf4j.sail.SailException: com.ontotext.trree.util.NotEnoughMemoryForDistinctGroupBy: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
        at org.eclipse.rdf4j.repository.sail.SailUpdate.execute(SailUpdate.java:69)
        at com.ontotext.trree.monitorRepository.MonitorRepositoryConnection$1.execute(MonitorRepositoryConnection.java:382)
        at org.eclipse.rdf4j.http.server.repository.statements.StatementsController.getSparqlUpdateResult(StatementsController.java:254)
        ... 45 common frames omitted
Caused by: org.eclipse.rdf4j.sail.SailException: com.ontotext.trree.util.NotEnoughMemoryForDistinctGroupBy: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
        at com.ontotext.trree.SailConnectionImpl.evaluateWithRepoConnectionInternal(SailConnectionImpl.java:1815)
        at com.ontotext.trree.SailConnectionImpl.evaluateWithRepoConnection(SailConnectionImpl.java:1679)
        at com.ontotext.trree.SailConnectionImpl.evaluateInternal(SailConnectionImpl.java:249)
        at com.ontotext.trree.SailConnectionImpl.evaluate(SailConnectionImpl.java:2316)
        at org.eclipse.rdf4j.repository.sail.helpers.SailUpdateExecutor.evaluateWhereClause(SailUpdateExecutor.java:487)
        at org.eclipse.rdf4j.repository.sail.helpers.SailUpdateExecutor.executeModify(SailUpdateExecutor.java:450)
        at org.eclipse.rdf4j.repository.sail.helpers.SailUpdateExecutor.executeUpdate(SailUpdateExecutor.java:121)
        at org.eclipse.rdf4j.repository.sail.SailUpdate.execute(SailUpdate.java:64)
        ... 47 common frames omitted
Caused by: com.ontotext.trree.util.NotEnoughMemoryForDistinctGroupBy: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
        at com.ontotext.trree.util.DistinctMemoryMonitor.updateGroupByMemory(DistinctMemoryMonitor.java:73)
        at com.ontotext.trree.util.LongKeyBase.<init>(LongKeyBase.java:38)
        at com.ontotext.trree.util.LongKeySet.<init>(LongKeySet.java:26)
        at com.ontotext.trree.query.OwlimDistinctIteration.<init>(OwlimDistinctIteration.java:20)
        at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:1086)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:639)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:228)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.TupleFunctionEvaluationStrategy.evaluate(TupleFunctionEvaluationStrategy.java:65)
        at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:135)
        at org.eclipse.rdf4j.query.algebra.evaluation.iterator.JoinIterator.<init>(JoinIterator.java:42)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:824)
        at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:237)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:796)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:230)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.TupleFunctionEvaluationStrategy.evaluate(TupleFunctionEvaluationStrategy.java:65)
        at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:135)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:650)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:228)
        at org.eclipse.rdf4j.query.algebra.evaluation.impl.TupleFunctionEvaluationStrategy.evaluate(TupleFunctionEvaluationStrategy.java:65)
        at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:135)
        at com.ontotext.trree.SailConnectionImpl.evaluateWithRepoConnectionInternal(SailConnectionImpl.java:1786)
        ... 54 common frames omitted
[INFO ] 2019-08-08 09:41:10,666 [repositories/kg-test-graph-import | c.o.g.s.StatementsController] POST SPARQL update request to repository

我们有哪些缓解情况的选择？ 目前，由于测试失败，我无法继续使用我们的软件来使其重新启动并运行。 这里的背景是，我们正在评估从另一个三重存储切换到Ontotext。

很多感谢和

最好的问候 汤姆

1 个答案:

答案 0 :(得分：1)

现在可以正常工作了。

这似乎是Docker Mac特有的问题。默认情况下，仅2GB内存分配给docker引擎。对此进行提高并为进程提供足够的内存（Xmx），现在为我解决了这个问题。

有关设置，请参见下面的屏幕截图。

由DockerIcon在那里->首选项->高级

Docker memory preferences

相关问题
最新问题