Question

如标题所述，如何将dir从一个位置递归复制到同一个s3存储桶中的另一个位置？我正在使用java api，我注意到类TransferMnager可以帮助从s3复制到s3，但我找不到允许递归复制dir的api。谁能帮助我？感谢

Answer 1

没有任何Java API可以递归复制。您可以将AWS CLI与--recursive标志一起使用，但它会遍历并复制所有文件。

此外，你可以实现类似的东西

public void run() throws ExecutionException {
    String bucketName = "someBucket";
    String folderPath = "folder/path";
    String sse = "AES256";
    setEncryptionForBucket(bucketName, folderPath, sse);
}

private void setEncryptionForBucket(String bucketName, String folderPath, String sse) throws ExecutionException {
    LOGGER.info("Setting encryption for: [" + bucketName + "] with encryption type: [" + sse + "] and " + (folderPath != null ? "with folder path: /" + folderPath : ""));
    AmazonS3 s3Client = factory.create(AmazonS3Client.class);
    TransferManager transferManager = TransferManagerBuilder.standard().build();
    List<String> failedKeys = new ArrayList<>();

    StopWatch stopWatch = new StopWatch();
    stopWatch.start();
    ParallelTaskExecutor parallelTaskExecutor = new ParallelTaskExecutor(THREAD_COUNT);
    s3Client.listObjectsV2(bucketName, folderPath)
            .getObjectSummaries().stream().map(S3ObjectSummary::getKey).forEach(key ->
            parallelTaskExecutor.submit(() -> setEncryptionForKey(transferManager, bucketName, key, failedKeys))
    );

    parallelTaskExecutor.waitForTasks();
    transferManager.shutdownNow();
    LOGGER.info("Task executed in " + stopWatch.getTime() / 1000 + " sec.");

    if (!failedKeys.isEmpty()) {
        LOGGER.error("Some of the keys cannot be encrypted: " + failedKeys);
        throw new RuntimeException("Some of the keys cannot be encrypted: " + failedKeys);
    }

    LOGGER.info("All objects encrypted successfully.");
}

private void setEncryptionForKey(TransferManager transferManager, String bucketName, String key, List<String> failedKeys) {
    CopyObjectRequest copyObjectRequest = new CopyObjectRequest(bucketName, key, bucketName, key);
    ObjectMetadata objectMetadata = new ObjectMetadata();
    objectMetadata.setSSEAlgorithm(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);
    copyObjectRequest.setNewObjectMetadata(objectMetadata);

    StopWatch stopWatch = new StopWatch();
    stopWatch.start();
    try {
        Copy xfer = transferManager.copy(copyObjectRequest);
        xfer.waitForCompletion();
        LOGGER.info("Xfer setting encryption successfully done for: " + key + " in " + stopWatch.getTime() + " ms.");
    } catch (Exception e) {
        LOGGER.error("Xfer failed for: " + key, e);
        failedKeys.add(key);
    }
}

aws将dir从一个位置递归复制到同一个s3存储桶中的另一个位置

1 个答案: