Question

这就是我写入InputStream

的方法

public OutputStream getOutputStream(@Nonnull final String uniqueId) throws PersistenceException {
        final PipedOutputStream outputStream = new PipedOutputStream();
        final PipedInputStream inputStream;
        try {
            inputStream = new PipedInputStream(outputStream);
            new Thread(
                    new Runnable() {
                        @Override
                        public void run() {
                            PutObjectRequest putObjectRequest = new PutObjectRequest("haritdev.sunrun", "sample.file.key", inputStream, new ObjectMetadata());
                            PutObjectResult result = amazonS3Client.putObject(putObjectRequest);
                            LOGGER.info("result - " + result.toString());
                            try {
                                inputStream.close();
                            } catch (IOException e) {

                            }
                        }
                    }
            ).start();
        } catch (AmazonS3Exception e) {
            throw new PersistenceException("could not generate output stream for " + uniqueId, e);
        } catch (IOException e) {
            throw new PersistenceException("could not generate input stream for S3 for " + uniqueId, e);
        }
         try {
            return new GZIPOutputStream(outputStream);
        } catch (IOException e) {
            LOGGER.error(e.getMessage(), e);
            throw new PersistenceException("Failed to get output stream for " + uniqueId + ": " + e.getMessage(), e);
        }
    }

并且在以下方法中，我看到我的进程死亡

protected <X extends AmazonWebServiceRequest> Request<X> createRequest(String bucketName, String key, X originalRequest, HttpMethodName httpMethod) {
    Request<X> request = new DefaultRequest<X>(originalRequest, Constants.S3_SERVICE_NAME);
    request.setHttpMethod(httpMethod);
    if (bucketNameUtils.isDNSBucketName(bucketName)) {
        request.setEndpoint(convertToVirtualHostEndpoint(bucketName));
        request.setResourcePath(ServiceUtils.urlEncode(key));
    } else {
        request.setEndpoint(endpoint);

        if (bucketName != null) {
            /*
             * We don't URL encode the bucket name, since it shouldn't
             * contain any characters that need to be encoded based on
             * Amazon S3's naming restrictions.
             */
            request.setResourcePath(bucketName + "/"
                    + (key != null ? ServiceUtils.urlEncode(key) : ""));
        }
    }

    return request;
}

该进程在 request.setResourcePath（ServiceUtils.urlEncode（key））; 上失败，我甚至无法进行调试，即使密钥是有效名称且不是 NULL

有人可以帮忙吗？

这是request在死亡前的样子

request = {com.amazonaws.DefaultRequest@1931}"PUT https://my.bucket.s3.amazonaws.com / "
resourcePath = null
parameters = {java.util.HashMap@1959} size = 0
headers = {java.util.HashMap@1963} size = 0
endpoint = {java.net.URI@1965}"https://my.bucket.s3.amazonaws.com"
serviceName = {java.lang.String@1910}"Amazon S3"
originalRequest = {com.amazonaws.services.s3.model.PutObjectRequest@1285}
httpMethod = {com.amazonaws.http.HttpMethodName@1286}"PUT"
content = null

Answer 1

我尝试了同样的方法，但对我来说也失败了。

我最后首先将所有数据写入输出流，然后在将数据从输出流复制到输入流后启动上传到S3：

...
// Data written to outputStream here
...
byte[] byteArray = outputStream.toByteArray();
amazonS3Client.uploadPart(new UploadPartRequest()
  .withBucketName(bucket)
  .withKey(key)
  .withInputStream(new ByteArrayInputStream(byteArray))
  .withPartSize(byteArray.length)
  .withUploadId(uploadId)
  .withPartNumber(partNumber));

如果整个数据块必须在上传到S3甚至开始之前在内存中写入和复制，那么写入流的目的就失败了，但这是我能让它工作的唯一方法。

Answer 2

这是我尝试和工作的内容 -

  try (PipedOutputStream pipedOutputStream = new PipedOutputStream();
     PipedInputStream pipedInputStream = new PipedInputStream()) {
            new Thread(new Runnable() {

            public void run() {
                try {
                      // write some data to pipedOutputStream
                } catch (IOException e) {
                   // handle exception
                }
            }
            }).start();
       PutObjectRequest putObjectRequest = new PutObjectRequest(BUCKET, FILE_NAME, pipedInputStream, new ObjectMetadata());
       s3Client.putObject(putObjectRequest);
}

此代码使用S3抛出警告，未设置内容长度，s3将被缓冲，并可能导致OutOfMemoryException。我不相信在ObjectMetadata中设置内容长度的任何廉价方法只是为了摆脱这个消息，并希望AWS SDK不会将整个流传输到内存中只是为了找到内容长度。

尝试将InputStream放入Amazon S3时，进程死亡

2 个答案: