我正在尝试使用HtmlUnit 2.11下载此页面：http://ekw.ms.gov.pl/pdcbdkw/pdcbdkw.html。在此页面上获取GET后，有JS脚本计算和设置cookie并提交表单。 我把标题看起来很像我的FF 18 - 我检查了它，用HtmlUnit的javascript计算的标题和cookie是正确的。我通过下载带脚本的页面并将表单中的操作更改为POST我自己的脚本shich打印请求标题和正文来检查它。我使用以下代码：
WebClient client = new WebClient();
client.setWebConnection(new HttpWebConnection(client) {
public WebResponse getResponse(WebRequest request) throws IOException {
if(request.getHttpMethod() == HttpMethod.POST) {
request.removeAdditionalHeader("Cache-Control");
}
else {
request.setAdditionalHeader("Cache-Control","max-age=0");
}
WebResponse response = super.getResponse(request);
return response;
}
});
client.addRequestHeader("User-Agent","Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0");
client.addRequestHeader("Host","ekw.ms.gov.pl");
client.addRequestHeader("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
client.addRequestHeader("Accept-Language","en-gb,en;q=0.5");
client.addRequestHeader("Accept-Encoding","gzip, deflate");
client.addRequestHeader("Connection", "keep-alive");
HtmlPage html = client.getPage("http://ekw.ms.gov.pl/pdcbdkw/pdcbdkw.html");
执行代码时会出现以下异常（在快捷方式中，它是由连接重置引起的）：
Exception in thread "main" ======= EXCEPTION START ========
Exception class=[java.lang.RuntimeException]
com.gargoylesoftware.htmlunit.ScriptException: Exception invoking submit
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:663)
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:559)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:525)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:594)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:569)
at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScriptFunctionIfPossible(HtmlPage.java:996)
at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeEventHandler(EventListenersContainer.java:208)
at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeBubblingListeners(EventListenersContainer.java:227)
at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:813)
at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:737)
at com.gargoylesoftware.htmlunit.html.HtmlElement$1.run(HtmlElement.java:853)
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:559)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:525)
at com.gargoylesoftware.htmlunit.html.HtmlElement.fireEvent(HtmlElement.java:858)
at com.gargoylesoftware.htmlunit.html.HtmlPage.executeEventHandlersIfNeeded(HtmlPage.java:1259)
at com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java:227)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:440)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:308)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:373)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:358)
at cenatorium.kwtest.App.main(App.java:49)
Caused by: java.lang.RuntimeException: Exception invoking submit
at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:163)
at net.sourceforge.htmlunit.corejs.javascript.FunctionObject.call(FunctionObject.java:452)
at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1473)
at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:815)
at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:109)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:415)
at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:274)
at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3132)
at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:107)
at com.gargoylesoftware.htmlunit.javascript.host.EventHandler.call(EventHandler.java:80)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$4.doRun(JavaScriptEngine.java:587)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:651)
... 20 more
Caused by: java.lang.RuntimeException: java.net.SocketException: Connection reset
at com.gargoylesoftware.htmlunit.WebClient.download(WebClient.java:2233)
at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLFormElement.submit(HTMLFormElement.java:310)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:137)
... 31 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:189)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:712)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:517)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:171)
at cenatorium.kwtest.App$1.getResponse(App.java:38)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1484)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1402)
at com.gargoylesoftware.htmlunit.WebClient.download(WebClient.java:2226)
... 37 more
Enclosed exception:
java.lang.RuntimeException: Exception invoking submit
at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:163)
at net.sourceforge.htmlunit.corejs.javascript.FunctionObject.call(FunctionObject.java:452)
at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1473)
at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:815)
at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:109)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:415)
at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:274)
at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3132)
at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:107)
at com.gargoylesoftware.htmlunit.javascript.host.EventHandler.call(EventHandler.java:80)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$4.doRun(JavaScriptEngine.java:587)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:651)
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:559)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:525)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:594)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:569)
at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScriptFunctionIfPossible(HtmlPage.java:996)
at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeEventHandler(EventListenersContainer.java:208)
at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeBubblingListeners(EventListenersContainer.java:227)
at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:813)
at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:737)
at com.gargoylesoftware.htmlunit.html.HtmlElement$1.run(HtmlElement.java:853)
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:559)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:525)
at com.gargoylesoftware.htmlunit.html.HtmlElement.fireEvent(HtmlElement.java:858)
at com.gargoylesoftware.htmlunit.html.HtmlPage.executeEventHandlersIfNeeded(HtmlPage.java:1259)
at com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java:227)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:440)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:308)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:373)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:358)
at cenatorium.kwtest.App.main(App.java:49)
Caused by: java.lang.RuntimeException: java.net.SocketException: Connection reset
at com.gargoylesoftware.htmlunit.WebClient.download(WebClient.java:2233)
at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLFormElement.submit(HTMLFormElement.java:310)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:137)
... 31 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:189)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:712)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:517)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:171)
at cenatorium.kwtest.App$1.getResponse(App.java:38)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1484)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1402)
at com.gargoylesoftware.htmlunit.WebClient.download(WebClient.java:2226)
... 37 more
== CALLING JAVASCRIPT ==
function () {
[native code, arity=0]
}
======= EXCEPTION END ========
以下是POST中发送的标题（与FF18相同，但顺序不同，但无关紧要）：
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0
Accept-Language: en-gb,en;q=0.5
Host: localhost
Referer: http://localhost/test.php
Accept-Encoding: gzip, deflate
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Content-Length: 69
Content-Type: application/x-www-form-urlencoded
Cookie: TS37535a_75=a538065ddccf975d9ae56e368a94eea0:hklh:781te27c:861731662
----------------- POST content
TS37535a_id=3&TS37535a_md=1&TS37535a_rf=0&TS37535a_ct=0&TS37535a_pd=0
cookie的值是正确的 - 我检查了它。
那么什么错？如何获取此页面 - 它应该返回html表单，您可以在浏览器中键入http://ekw.ms.gov.pl/pdcbdkw/pdcbdkw.html来查看...
答案 0 :(得分：0)
在2晚之后写了我自己的http客户端并发送原始请求我想出来了：
在POST请求中，数据的发送顺序与以html格式显示的顺序不同：
键2 = val2的＆安培;键1 = VAL1＆安培; KEY3 = VAL3
应该是
key1=val1&key2=val2&key3=val3
它不应该有问题，但它适用于此服务器。
connection reset异常，我必须重新连接并重试请求。