我需要同时将多个文件上传到AWS S3。我使用ExecuterService在不同的线程上上传文件。当我启动arround 100个线程时，它开始抛出异常：
com.amazonaws.SdkClientException: Unable to execute HTTP request: null
| at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1114) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1064) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4325) ~[aws-java-sdk-s3-1.11.248.jar!/:na]
| at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4272) ~[aws-java-sdk-s3-1.11.248.jar!/:na]
| at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1749) ~[aws-java-sdk-s3-1.11.248.jar!/:na]
| ... 9 common frames omitted
| Caused by: java.nio.channels.ClosedChannelException: null
| at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110) ~[na:1.8.0_171]
| at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:147) ~[na:1.8.0_171]
| at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65) ~[na:1.8.0_171]
| at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109) ~[na:1.8.0_171]
| at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103) ~[na:1.8.0_171]
| at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:82) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:82) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.util.LengthCheckInputStream.read(LengthCheckInputStream.java:107) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:82) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream.read(MD5DigestCalculatingInputStream.java:128) ~[aws-java-sdk-s3-1.11.248.jar!/:na]
| at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[na:1.8.0_171]
| at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[na:1.8.0_171]
| at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[na:1.8.0_171]
| at com.amazonaws.internal.SdkBufferedInputStream.read(SdkBufferedInputStream.java:76) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:82) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:180) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:82) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at org.apache.http.entity.InputStreamEntity.writeTo(InputStreamEntity.java:140) ~[httpcore-4.4.9.jar!/:4.4.9]
| at com.amazonaws.http.RepeatableInputStreamRequestEntity.writeTo(RepeatableInputStreamRequestEntity.java:160) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156) ~[httpcore-4.4.9.jar!/:4.4.9]
| at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:160) ~[httpclient-4.5.4.jar!/:4.5.4]
| at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238) ~[httpcore-4.4.9.jar!/:4.4.9]
| at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doSendRequest(SdkHttpRequestExecutor.java:63) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123) ~[httpcore-4.4.9.jar!/:4.4.9]
| at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.4.jar!/:4.5.4]
| at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.4.jar!/:4.5.4]
| at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.4.jar!/:4.5.4]
| at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.4.jar!/:4.5.4]
| at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.4.jar!/:4.5.4]
| at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1236) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056) ~[aws-java-sdk-core-1.11.248.jar!/:na]
| ... 20 common frames omitted
我使用的Java AWS Sdk版本是1.11.248。
ExecuterSErvice用作：
ExecutorService ioThreadPool = Executors.newFixedThreadPool(100);
ConcurrentLinkedDeque<Future<?>> uploadTasks = new ConcurrentLinkedDeque<>();
Future<?> future = ioThreadPool.submit(() -> {
try {
putObjectToS3();
} catch (IOException e) {
throw new InternalErrorException("IOException during file upload.", e);
}
});
uploadTasks.push(future);
CompletableFuture.allOf(uploadTasks.toArray(new CompletableFuture<?>[0])).join();
我还将AWS Client中的maxNumberOfConnection设置为150
由于