TransferManager卡在大文件传输上

时间:2019-02-06 22:16:27

标签: java amazon-web-services amazon-s3

我正在编写使用AWS开发工具包TransferManager类将文件上传到AWS S3的代码。但是,它似乎总是在特定文件上触发。该文件约为19MB,并具有640的权限(本质上,我没有对该文件的读取访问权限)。但是,它似乎一直陷于无限循环中,在该循环中,传输的字节数从未增加,并且没有引发任何事件。

代码本身可在其他640个文件上使用。我收到“权限被拒绝”错误,但是代码正常退出。在这种情况下,代码永远不会终止。这是代码以及上载的事件日志。在这种情况下,有什么办法可以终止上传?这一直持续到我用完堆空间为止。

public void transferFiles() throws InterruptedException {       
    File directoryToUpload = getDirectoryToUpload();
    final AmazonS3 client = this.client();
    final TransferManager b = new TransferManager(client);

    TransferManagerConfiguration txc = new TransferManagerConfiguration();
    txc.setMultipartUploadThreshold(5 * 1024 * 1024);
    txc.setMinimumUploadPartSize(5 * 1024 * 1024);
    b.setConfiguration(txc);

    System.out.println("Uploading " + getDirectoryName());
    final MultipleFileUpload multiUpload = b.uploadDirectory(getBucketName(), getDirectoryName(), getDirectoryToUpload(), true);

    Collection<? extends Upload> subtransfers = multiUpload.getSubTransfers();
    subtransfers = multiUpload.getSubTransfers();

    do {
        for (final Upload upload : subtransfers) {
            String description = upload.getDescription();
            final String fileName = description.substring(description.indexOf(getPrefix()) + getPrefix().length() - 1);

            upload.addProgressListener(new ProgressListener() {

                @Override
                public void progressChanged(ProgressEvent event) {

                    System.out.println(event.getEventType());
                    if (event.getEventType() == ProgressEventType.TRANSFER_COMPLETED_EVENT) {
                        System.out.println("Completed transfer for file: " + fileName);
                    } else if (event.getEventType() == ProgressEventType.TRANSFER_PART_FAILED_EVENT ||
                            event.getEventType() == ProgressEventType.TRANSFER_FAILED_EVENT ||
                            event.getEventType() == ProgressEventType.CLIENT_REQUEST_FAILED_EVENT) {
                        System.out.println("Failed transferring file: " + fileName);
                    } else if (event.getEventType() == ProgressEventType.CLIENT_REQUEST_RETRY_EVENT) {
                        System.out.println("Retrying to upload file: " + fileName);
                    }
                }

            });

            do {
                System.out.println("Transfer details for = " + fileName);
                System.out.println(upload.getProgress().getBytesTransferred() + " out of " + upload.getProgress().getTotalBytesToTransfer() + " done.");
                Thread.sleep(1000);
            } while (upload.getProgress().getBytesTransferred() < upload.getProgress().getTotalBytesToTransfer());

        }

        try {
            Thread.sleep(5000);
        } catch (InterruptedException e) {
            System.out.println("Exception encountered: " + e.getMessage());
            e.printStackTrace();
        }
    } while (!multiUpload.isDone());

    System.out.println("... " + multiUpload.getState());

    multiUpload.waitForCompletion();

    b.shutdownNow();
}

输出:

Transfer details for = /my/large/file.csv
0 out of 19501727 done.
TRANSFER_STARTED_EVENT
CLIENT_REQUEST_STARTED_EVENT
HTTP_REQUEST_STARTED_EVENT
HTTP_REQUEST_COMPLETED_EVENT
HTTP_RESPONSE_STARTED_EVENT
RESPONSE_BYTE_TRANSFER_EVENT
HTTP_RESPONSE_COMPLETED_EVENT
CLIENT_REQUEST_SUCCESS_EVENT
Transfer details for = /my/large/file.csv
524 out of 19501727 done.
TRANSFER_FAILED_EVENT
Failed transferring file: /my/large/file.csv
Transfer details for = /my/large/file.csv
524 out of 19501727 done.
Transfer details for = /my/large/file.csv
524 out of 19501727 done.
Transfer details for = /my/large/file.csv
524 out of 19501727 done.
Transfer details for = /my/large/file.csv
524 out of 19501727 done.
Transfer details for = /my/large/file.csv
524 out of 19501727 done.
Transfer details for = /my/large/file.csv
524 out of 19501727 done.
Transfer details for = /my/large/file.csv
524 out of 19501727 done.
Transfer details for = /my/large/file.csv
524 out of 19501727 done.
Transfer details for = /my/large/file.csv
524 out of 19501727 done.
Transfer details for = /my/large/file.csv
524 out of 19501727 done.
Transfer details for = /my/large/file.csv
524 out of 19501727 done.
Transfer details for = /my/large/file.csv
524 out of 19501727 done.

0 个答案:

没有答案