我正在运行一个ThreadPoolExecutor程序,它创建1000个url并发送给执行程序服务。
Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:673)
at java.util.concurrent.ThreadPoolExecutor.addThread(ThreadPoolExecutor.java:681)
at java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:706)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:650)
at MyProgramName.main(MyProgramName.java:175)
Aug 16, 2014 8:46:20 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {}->MyURLURLURLURLRURL Connection reset
Aug 16, 2014 8:46:20 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {}->MyURLURLURLURLRURL Connection reset
Aug 16, 2014 8:46:20 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->MyURLURLURLURLRURL
Aug 16, 2014 8:46:20 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->MyURLURLURLURLRURL
2014-08-16 20:46:21.341 java[5100:d07] java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:673)
at sun.awt.AWTAutoShutdown.activateBlockerThread(AWTAutoShutdown.java:312)
at sun.awt.AWTAutoShutdown.setToolkitBusy(AWTAutoShutdown.java:232)
at sun.awt.AWTAutoShutdown.notifyToolkitThreadBusy(AWTAutoShutdown.java:118)
2014-08-16 20:46:21.342 java[5100:d07] (
0 CoreFoundation 0x00007fff9127825c __exceptionPreprocess + 172
1 libobjc.A.dylib 0x00007fff9031ce75 objc_exception_throw + 43
2 CoreFoundation 0x00007fff91277e09 -[NSException raise] + 9
3 JavaNativeFoundation 0x000000010c754c3f JNFCallStaticVoidMethod + 213
4 libawt.jnilib 0x0000000115fb8151 setBusy + 53
5 CoreFoundation 0x00007fff911a8d67 __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23
6 CoreFoundation 0x00007fff911a8cd7 __CFRunLoopDoObservers + 391
7 CoreFoundation 0x00007fff9119a608 __CFRunLoopRun + 1368
8 CoreFoundation 0x00007fff91199e75 CFRunLoopRunSpecific + 309
9 HIToolbox 0x00007fff8fdd9a0d RunCurrentEventLoopInMode + 226
10 HIToolbox 0x00007fff8fdd97b7 ReceiveNextEventCommon + 479
11 HIToolbox 0x00007fff8fdd95bc _BlockUntilNextEventMatchingListInModeWithFilter + 65
12 AppKit 0x00007fff8ac5d24e _DPSNextEvent + 1434
13 AppKit 0x00007fff8ac5c89b -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 122
14 libawt.jnilib 0x0000000115fb987f -[NSApplicationAWT nextEventMatchingMask:untilDate:inMode:dequeue:] + 124
15 AppKit 0x00007fff8ac5099c -[NSApplication run] + 553
16 libawt.jnilib 0x0000000115fb7f30 +[AWTStarter startAWT:] + 1495
17 libawt.jnilib 0x0000000115fb78aa -[CPerformer perform] + 93
18 Foundation 0x00007fff8c64313e __NSThreadPerformPerform + 229
19 CoreFoundation 0x00007fff911a95b1 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
20 CoreFoundation 0x00007fff9119ac62 __CFRunLoopDoSources0 + 242
21 CoreFoundation 0x00007fff9119a3ef __CFRunLoopRun + 831
22 CoreFoundation 0x00007fff91199e75 CFRunLoopRunSpecific + 309
23 java 0x000000010be86843 java + 18499
24 java 0x000000010be8629a java + 17050
25 java 0x000000010be83a98 java + 6808
26 ??? 0x0000000000000005 0x0 + 5
)
Aug 16, 2014 8:46:26 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {}->MyURLURLURLURLRURL Connection reset
Aug 16, 2014 8:46:26 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->MyURLURLURLURLRURL
Aug 16, 2014 8:46:26 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {}->MyURLURLURLURLRURL Connection reset
Aug 16, 2014 8:46:26 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->MyURLURLURLURLRURL
Aug 16, 2014 8:46:26 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {}->MyURLURLURLURLRURL Connection reset
Aug 16, 2014 8:46:26 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->MyURLURLURLURLRURL
Aug 16, 2014 8:46:26 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {}->MyURLURLURLURLRURL Connection reset
Aug 16, 2014 8:46:26 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {}->MyURLURLURLURLRURL Connection reset
Aug 16, 2014 8:46:26 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->MyURLURLURLURLRURL
我已经在互联网和stackoverflow上做了一些研究。一些答案说我需要减少堆栈大小。有人说我需要使用ulimit -a来增加ulimit。我完全糊涂了。我对这些答案感到困惑,因为它们并不特定于ThreadExecutor。请帮帮我
我在Mac OS,i7四核处理器,16 GB RAM,1 Tb HD中运行代码
我的计划代码在这个问题Why does the following executor service java Thread program doesn't shut down?
中答案 0 :(得分:4)
真正的问题是你正在尝试使用过多的堆栈内存来创建太多的线程。
您可以减少线程堆栈的默认大小(如建议的那样),但这不是一个完整的解决方案。 (当你决定在更多网址上运行你的程序时,你会再次遇到同样的问题。最终你会遇到“墙”,你无法进一步减少堆栈大小;例如,因为你得到{{1例外。)
更好的解决方案是不要创建这么多线程。您已将程序配置为使用无限线程池。这坦率地说是荒谬的。改变这个:
StackOverflowError
更现实:
ThreadPoolExecutor executor =
new ThreadPoolExecutor(Integer.MAX_VALUE, Integer.MAX_VALUE,
20, TimeUnit.MILLISECONDS, blockingQueue);
超过某一点,添加额外的线程不会使应用程序更快。实际上,它通常会开始使应用程序变慢,有时灾难性更慢。
答案 1 :(得分:0)
当JVM从操作系统询问新线程时,您将面临“java.lang.OutOfMemoryError:无法创建新的本机线程”。只要底层操作系统无法分配新的本机线程,就会抛出此OutOfMemoryError。本机线程的确切限制是非常依赖于平台的,因此我们建议通过运行一个小测试找出您的限制,以找出特定于平台的线程限制。但是,通常情况下,导致 java.lang.OutOfMemoryError:无法创建新的本机线程的情况会经历以下阶段:
在您的情况下,克服该问题的最佳方法是遵循建议的限制线程创建的路径。