apache 2.4 lot of stuck threads in "W" sending reply mode

Question

I just upgraded Apache webserver from 2.2 to 2.4. After upgrade, I am getting lot of stuck threads in W mode. I am not using php. The stuck threads are happening even on static html pages or while doing load testing via Jmeter.

Because of these stuck threads, it is reaching the MaxRequestWorkers/MaxClients Limit and server becomes non-responsive. Memory is not an issue. Since even during the server crash I had about 10Gb of free memory.

Just to verify if it is Apache, I swithced it back to 2.2 (old server) and no more hung threads!

ServerTokens OS ServerRoot "/etc/httpd" PidFile run/httpd.pid TimeOut 295 KeepAlive Off MaxKeepAliveRequests 100 KeepAliveTimeout 15

StartServers 20 MinSpareServers 15 MaxSpareServers 40 ServerLimit 1024

MaxClients 2048

MaxRequestWorkers 2048

MaxRequestsPerChild 5000

MaxConnectionsPerChild 5000

Following is the pstack on one fo the threads:

pstack 30078

#0  0x00007f0c6536df4d in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007f0c65369d02 in _L_lock_791 () from /lib64/libpthread.so.0
#2  0x00007f0c65369c08 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00007f0c58e53f4b in yodysMtxLock () from /etc/httpd/modules/libdms2.so
#4  0x00007f0c58e518e6 in yodSlotLock () from /etc/httpd/modules/libdms2.so
#5  0x00007f0c58e50266 in yodStateIncrementSb8 () from        /etc/httpd/modules/libdms2.so
#6  0x00007f0c597ca764 in wl_increment_state_metric () from /etc/httpd/modules/mod_wl_24.so
#7  0x00007f0c597c5af9 in request_handler () from /etc/httpd/modules/mod_wl_24.so
#8  0x00007f0c668dc290 in ap_run_handler ()
#9  0x00007f0c668dc7d9 in ap_invoke_handler ()
#10 0x00007f0c668f0bca in ap_process_async_request ()
#11 0x00007f0c668f0ea4 in ap_process_request ()
#12 0x00007f0c668ed7f2 in ap_process_http_connection ()
#13 0x00007f0c668e5890 in ap_run_process_connection ()
#14 0x00007f0c58c2280f in child_main () from /etc/httpd/modules/mod_mpm_prefork.so
#15 0x00007f0c58c22a55 in make_child () from /etc/httpd/modules/mod_mpm_prefork.so
#16 0x00007f0c58c22ab6 in startup_children () from /etc/httpd/modules/mod_mpm_prefork.so
#17 0x00007f0c58c237c0 in prefork_run () from  /etc/httpd/modules/mod_mpm_prefork.so
#18 0x00007f0c668c25be in ap_run_mpm ()
#19 0x00007f0c668bbb46 in main ()

Any guidance will be helpful.

Answer 1

特定线程卡在等待锁的专有weblogic插件中。令人惊讶的是它甚至可以触发静态请求。但是，您需要与供应商合作，没有其他人可以调试它。