使用HtmlUnit在java中出现内存不足错误

时间：2016-05-30 22:42:23

标签： java htmlunit

我使用htmlUnit和java构建了一个web机器人，它可以执行大量的操作，但是在运行了大约10分钟后，我总是会遇到以下错误。

该程序处理一些文本文件，但下面的错误似乎与HtmlUnit解析javascript有关。 

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at net.sourceforge.htmlunit.corejs.javascript.Parser.forLoop(Parser.java:1297)
at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:982)
at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:926)
at net.sourceforge.htmlunit.corejs.javascript.Parser.statements(Parser.java:886)
at net.sourceforge.htmlunit.corejs.javascript.Parser.block(Parser.java:1699)
at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:1031)
at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:926)
at net.sourceforge.htmlunit.corejs.javascript.Parser.ifStatement(Parser.java:1102)
at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:970)
at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:926)
at net.sourceforge.htmlunit.corejs.javascript.Parser.parseFunctionBody(Parser.java:662)
at net.sourceforge.htmlunit.corejs.javascript.Parser.function(Parser.java:827)
at net.sourceforge.htmlunit.corejs.javascript.Parser.primaryExpr(Parser.java:2746)
at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExpr(Parser.java:2388)
at net.sourceforge.htmlunit.corejs.javascript.Parser.unaryExpr(Parser.java:2292)
at net.sourceforge.htmlunit.corejs.javascript.Parser.mulExpr(Parser.java:2219)
at net.sourceforge.htmlunit.corejs.javascript.Parser.addExpr(Parser.java:2205)
at net.sourceforge.htmlunit.corejs.javascript.Parser.shiftExpr(Parser.java:2188)
at net.sourceforge.htmlunit.corejs.javascript.Parser.relExpr(Parser.java:2165)
at net.sourceforge.htmlunit.corejs.javascript.Parser.eqExpr(Parser.java:2139)
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitAndExpr(Parser.java:2130)
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitXorExpr(Parser.java:2121)
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitOrExpr(Parser.java:2112)
at net.sourceforge.htmlunit.corejs.javascript.Parser.andExpr(Parser.java:2103)
at net.sourceforge.htmlunit.corejs.javascript.Parser.orExpr(Parser.java:2094)
at net.sourceforge.htmlunit.corejs.javascript.Parser.condExpr(Parser.java:2060)
at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2033)
at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2044)
at net.sourceforge.htmlunit.corejs.javascript.Parser.expr(Parser.java:2023)
at net.sourceforge.htmlunit.corejs.javascript.Parser.nameOrLabel(Parser.java:1774)
at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:1053)
at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:926)

我在使用HtmlUnit加载javascript页面时会出现大量错误，例如：

net.sourceforge.htmlunit.corejs.javascript.EcmaError: TypeError: Expected argument of type object, but instead had type object (http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js#4)
at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3935)
at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3919)
at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.typeError(ScriptRuntime.java:3944)
at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.typeError1(ScriptRuntime.java:3954)
at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.ensureScriptableObject(ScriptableObject.java:2193)
at net.sourceforge.htmlunit.corejs.javascript.NativeObject.execIdCall(NativeObject.java:303)
at net.sourceforge.htmlunit.corejs.javascript.IdFunctionObject.call(IdFunctionObject.java:93)
at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1536)
at script(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:4)
at script(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:1)
at script.n(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:1)
at script(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:5)
at script(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:1)
at script.n(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:1)
at script(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:4)
at script.n(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:1)
at script(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:3)
at script(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:3)
at script.n(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:1)
at script(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:4)
at script.n(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:1)
at script(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:4)
at script.n(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:1)
at script(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:1)
at script.n(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:1)
at script(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:1)
at script(http://ca.classistatic.com/service-static/frontend-service/common.b28d6b2c3f10a7b5388a.js:1)
at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:798)
at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:411)
at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:309)
at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3286)
at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:115)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$3.doRun(JavaScriptEngine.java:827)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:939)
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:628)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:513)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:836)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:812)
at com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:997)
at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:399)
at com.gargoylesoftware.htmlunit.html.HtmlScript$3.execute(HtmlScript.java:277)
at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:293)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:799)
at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:756)
at org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1170)
at org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1072)
at org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206)
at org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:330)
at org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3126)
at org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2093)
at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:920)
at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499)
at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:1039)
at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:252)
at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:198)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:271)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:159)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:478)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:352)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:417)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:402)
at WebBotServicesAdID.replyToAd(WebBotServicesAdID.java:269)
at WebBotServicesAdID.main(WebBotServicesAdID.java:771)
======= EXCEPTION END ========

然而，这些错误似乎是不可避免的，并且它们不会导致程序停止。页面仍然被加载，我仍然可以做我需要做的事情，所以我总是忽略它们。

这个OutOfMemoryError是我忽略的那些错误累积的结果吗？

1 个答案:

答案 0 :(得分：0)

GC错误：

GC代表垃圾收集器，错误意味着GC正在努力工作，几乎什么都不做。例如，已经讨论过here和here这样的错误。您可以尝试增加堆（“-Xmx1024m”，“ - Xmx512m”等）或禁用错误验证（“-XX：-UseGCOverheadLimit”）。我认为你不应该使用其中任何一种选择。

如果你正在运行linux，你可以尝试使用jmap来检查堆中的实时数据，然后相应地优化你的代码。

JS错误：

根据this回答，这些错误似乎是由脚本错误引起的。尝试给出的建议来禁用它们。

如果抛出这些错误并且未正确处理，则某些对象可能会留在堆中导致GC问题。或者你可能有一些代码（例如一个循环）实例化更多的对象。

相关问题
最新问题