Google Compute Engine套接字问题

时间:2015-12-30 16:06:59

标签: tomcat debian google-compute-engine google-cloud-platform google-cloud-debugger

最近几天,Debian(Google Compute Engine)服务器每天都面临套接字不可用的问题,需要重启tomcat。

Dec 30, 2015 1:16:53 PM org.apache.tomcat.util.net.JIoEndpoint$Acceptor run
SEVERE: Socket accept failed
java.net.SocketException: Too many open files
    at java.net.PlainSocketImpl.socketAccept(Native Method)
    at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
    at java.net.ServerSocket.implAccept(ServerSocket.java:530)
    at java.net.ServerSocket.accept(ServerSocket.java:498)
    at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:60)
    at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:220)
    at java.lang.Thread.run(Thread.java:745)

在使用netstat -ap检查的情况下使用时发现GOOGLE :(正在充斥服务器。

tcp        1      0 backend-tomcat:53976 wk-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:42251 wl-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:45929 wn-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:42159 wl-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:55348 wo-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:44973 wa-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:42148 wl-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:45729 wn-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:45721 wn-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:42146 wl-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:36557 wq-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:45723 wn-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:45915 wn-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:45295 wn-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:53819 wk-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        0      0 backend-tomcat:45968 wn-in-f95.1e100.n:https ESTABLISHED 5081/java       
tcp        1      0 backend-tomcat:41737 wl-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:55132 wm-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:53969 wk-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:40883 wj-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:45734 wn-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:40889 wj-in-f95.1e100.n:https CLOSE_WAIT  5081/java       
tcp        1      0 backend-tomcat:41834 wl-in-f95.1e100.n:https CLOSE_WAIT  5081/java       

使用lsof -p 5081检查了流程,下面有大量的行列表

java    5081 tomcat7 1181u  IPv4            1959257      0t0     TCP backend-tomcat.c.stutzenappointments.internal:48570->wb-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1182u  IPv4            1959420      0t0     TCP backend-tomcat.c.stutzenappointments.internal:48575->wb-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1183u  IPv4            1959526      0t0     TCP backend-tomcat.c.stutzenappointments.internal:48577->wb-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1184u  IPv4            1959676      0t0     TCP backend-tomcat.c.stutzenappointments.internal:48583->wb-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1185u  IPv4            1960099      0t0     TCP backend-tomcat.c.stutzenappointments.internal:48591->wb-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1186u  IPv4            1959857      0t0     TCP backend-tomcat.c.stutzenappointments.internal:48589->wb-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1187u  IPv4            1960209      0t0     TCP backend-tomcat.c.stutzenappointments.internal:48596->wb-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1188u  IPv4            1960352      0t0     TCP backend-tomcat.c.stutzenappointments.internal:49524->wa-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1189u  IPv4            1960555      0t0     TCP backend-tomcat.c.stutzenappointments.internal:49526->wa-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1190u  IPv4            1960721      0t0     TCP backend-tomcat.c.stutzenappointments.internal:49531->wa-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1191u  IPv4            1960962      0t0     TCP backend-tomcat.c.stutzenappointments.internal:49537->wa-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1192u  IPv4            1961151      0t0     TCP backend-tomcat.c.stutzenappointments.internal:49539->wa-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1193u  IPv4            1961318      0t0     TCP backend-tomcat.c.stutzenappointments.internal:49544->wa-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1194u  IPv4            1961557      0t0     TCP backend-tomcat.c.stutzenappointments.internal:49550->wa-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1195u  IPv4            1961828      0t0     TCP backend-tomcat.c.stutzenappointments.internal:46380->wl-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1196u  IPv4            1962155      0t0     TCP backend-tomcat.c.stutzenappointments.internal:46386->wl-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1197u  IPv4            1962356      0t0     TCP backend-tomcat.c.stutzenappointments.internal:46392->wl-in-f95.1e100.net:https (CLOSE_WAIT)
java    5081 tomcat7 1198u  IPv4            1962461      0t0     TCP backend-tomcat.c.stutzenappointments.internal:46393->wl-in-f95.1e100.net:https (CLOSE_WAIT)

从ulimit发现打开文件限制设置为max

open files                      (-n) 65536

是否归功于GCE的云调试器功能?有没有办法要求Google不要每天让服务器流血?

1 个答案:

答案 0 :(得分:2)

感谢您报告。此错误现已修复:https://github.com/GoogleCloudPlatform/cloud-debug-java/commit/057d2f181ebfb5b670e31ecc9c155b38cc495150

请在format_env_gce.sh脚本中删除应用程序版本,以确保VM正在获取最新版本的调试器代理。例如:

...format_env_gce.sh --version=2 ...

如果您使用Managed VM,则下次重新部署时它将自动更新为最新版本的调试器代理。

项目的GitHub page提供了最新的二进制文件。