Question

我有一个安静的网络服务，它发送一个zip文件作为输出。 Restful Web服务是使用Apache CXF开发的。该服务部署在AWS Elastic Load平衡器后面。 70％的时间服务正常运行。有时我们会得到以下提到的错误

Sending the response
04-Dec-2015 13:59:25.844 WARNING [http-nio-8080-exec-3] org.apache.cxf.phase.PhaseInterceptorChain.doDefaultLogging Interceptor for {http://webservices.heavywater.solutions/}HocrServicesImpl has thrown exception, unwinding now
 org.apache.cxf.interceptor.Fault: Could not send Message.
    at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:64)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:272)
    at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:83)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:272)
    at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
    at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:241)
    at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:248)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:222)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:153)
    at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:171)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:286)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:206)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:262)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:617)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:668)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1521)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1478)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:393)
    at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:426)
    at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:342)
    at org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:295)
    at org.apache.catalina.connector.CoyoteOutputStream.close(CoyoteOutputStream.java:151)
    at org.apache.cxf.transport.http.AbstractHTTPDestination$WrappedOutputStream.close(AbstractHTTPDestination.java:749)
    at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
    at org.apache.cxf.transport.http.AbstractHTTPDestination$BackChannelConduit.close(AbstractHTTPDestination.java:679)
    at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
    ... 34 more
Caused by: java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:65)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:492)
    at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:127)
    at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
    at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:173)
    at org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:139)
    at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:197)
    at org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:41)
    at org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:320)
    at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:118)
    at org.apache.coyote.http11.AbstractOutputBuffer.doWrite(AbstractOutputBuffer.java:256)
    at org.apache.coyote.Response.doWrite(Response.java:503)
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:388)
    ... 42 more

04-Dec-2015 13:59:25.948 WARNING [http-nio-8080-exec-3] org.apache.cxf.phase.PhaseInterceptorChain.doDefaultLogging Interceptor for {http://webservices.heavywater.solutions/}HocrServicesImpl has thrown exception, unwinding now
 org.apache.cxf.interceptor.Fault: Could not send Message.
    at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:64)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:272)
    at org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:113)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:336)
    at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:83)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:272)
    at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
    at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:241)
    at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:248)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:222)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:153)
    at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:171)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:286)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:206)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:262)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:617)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:668)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1521)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1478)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:393)
    at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:426)
    at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:342)
    at org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:295)
    at org.apache.catalina.connector.CoyoteOutputStream.close(CoyoteOutputStream.java:151)
    at org.apache.cxf.transport.http.AbstractHTTPDestination$WrappedOutputStream.close(AbstractHTTPDestination.java:749)
    at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
    at org.apache.cxf.transport.http.AbstractHTTPDestination$BackChannelConduit.close(AbstractHTTPDestination.java:679)
    at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
    ... 36 more
Caused by: java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:65)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:492)
    at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:127)
    at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
    at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:173)
    at org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:139)
    at org.apache.coyote.http11.InternalNioOutputBuffer.flushBuffer(InternalNioOutputBuffer.java:244)
    at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:189)
    at org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:41)
    at org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:320)
    at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:116)
    at org.apache.coyote.http11.AbstractOutputBuffer.doWrite(AbstractOutputBuffer.java:256)
    at org.apache.coyote.Response.doWrite(Response.java:503)
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:388)
    ... 44 more

这是用于发送响应的服务器端代码。

System.out.println("Sending the response");
File file = new File(folderPath+"/zipFile.zip");
ResponseBuilder builder = Response.ok((Object)file);
builder.header("Content-Type", "application/zip");
builder.header("Content-Disposition", "attachment; filename=\"zipFile.zip\"");
return builder.build();

客户端代码

RequestEntity entity = new InputStreamRequestEntity(stream);
            methodHeader.setName("content-type");
            methodHeader.setValue("application/octet-stream");
            methodHeader.setName("accept");
            methodHeader.setValue("application/zip");
            methodHeader.setName("content-length");
            methodHeader.setValue(fileSize);;
            mPost.addRequestHeader(methodHeader);
            mPost.setRequestEntity(entity);

            int statusCode = httpClient.executeMethod(mPost);

            System.out.println(statusCode);
            if(statusCode == 200){
                InputStream is = mPost.getResponseBodyAsStream();
                f = new File("/tmp/"+tiffFileName.replace(".tif", "_input.zip"));
                FileOutputStream fos = new FileOutputStream(f);
                try{
                    byte[] buf = new byte[1024];
                    int len = is.read(buf);
                    while(len>0){
                        fos.write(buf, 0, len);
                        len = is.read(buf);
                    }           
                }finally{
                    if(fos != null)
                        fos.close();
                }
            }else if(statusCode == 403){
                System.out.println("Invalid username/password");
            }else{
                System.out.println(mPost.getResponseBodyAsString());
            }


        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }finally{
            if(mPost !=null){
                mPost.releaseConnection();
            }
        }

Answer 1

By default Elastic load balancer idle timeout is 60 seconds. If your Web service request takes more than 60 second then connection will be timed out and you will get the broken pipe exception. To Resolve this issue make the following changes in the ELB

Increase the Idle timeout under ELB settings > Descriptions > Connection Settings
Increase the connection timeout in the tomcat server config files (server.xml)

<Connector port="8080" redirectPort="8443" connectionTimeout="900000" protocol="HTTP/1.1"/>

Hope this will resolve the issues.

Answer 2

我也面临着管道破裂的问题，并且很难破解。问题是断断续续的，仅在AWS环境中发生，并且在本地和本地服务器上都能很好地工作。

当在交换头中设置的内容的大小很大时，就会出现问题。通过将内容设置为交换的“属性”而不是“标题”来解决该问题。它在非Cloud环境中工作的原因是Tomcat标头的最大大小（server.max-http-header-size）显着增加了，但是相同的Web服务器设置却没有在Cloud中使用。

希望对其他遇到类似问题的人有帮助。

ELB后面的Apache CXF中的管道异常断开

2 个答案: