Question

我是Amazon s3 Web服务的新手，需要开发一个命令行应用程序来在Amazon S3存储桶之间传输文件。必须将输入文件的内容转换为目标格式，然后将其复制到目标文件夹。目标格式可以是XML或Json，文件内容可以是给定的数据模型。

我有Java的中级经验，只是创建了一个尚待处理的帐户，因此，尝试开发一个工作流来解决问题。

Answer 1

嗯，这并不难。我几个月前就已经为客户做过了，你可能会在下面找到代码。要从AmazonS3存储桶中读取文件，请阅读此亚马逊文档[1]。要将文件写入Amazon s3存储桶，请阅读本文档[2]。

除此之外，您可能需要将所有访问令牌添加到本地操作系统中。您可以从管理员那里获得一些帮助。获得正确的凭据是我记忆中唯一棘手的部分。

亚马逊有一个很好的小文档，我建议你也经历这个。

package org.saig.watermark.demo;

import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.FilterInputStream;
import java.io.IOException;
import java.net.URL;

import org.apache.commons.io.IOUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import com.amazonaws.AmazonClientException;
import com.amazonaws.HttpMethod;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.regions.Region;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.GeneratePresignedUrlRequest;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.model.S3Object;

public class AmazonS3Util {
    private static AWSCredentials credentials = null;
    private static final String fileSeparator = "/";
    private static final Log log = LogFactory.getLog(AmazonS3Util.class);
    static {
        /*
         * The ProfileCredentialsProvider will return your [default]
         * credential profile by reading from the credentials file located at
         * (~/.aws/credentials).
         */
        try {
            credentials = new ProfileCredentialsProvider().getCredentials();
        } catch (Exception e) {
            throw new AmazonClientException(
                                            "Cannot load the credentials from the credential profiles file. "
                                                    + "Please make sure that your credentials file is at the correct "
                                                    + "location (~/.aws/credentials), and is in valid format.",
                                            e);
        }
    }

    public static void readFileFromS3cketBucket(String bucketName, String key, String dirPath,
                                                String fileName) {
        FilterInputStream inputStream = null;
        FileOutputStream outputStream = null;
        try {
            // Remove the file if it already exists.
            if (new File(dirPath + WatermarkConstants.fileSeparator + fileName).exists()) {
                FileUtil.delete(new File(dirPath + WatermarkConstants.fileSeparator + fileName));
            }

            AmazonS3 s3 = new AmazonS3Client(credentials);
            Region usEast1 = Region.getRegion(Regions.US_EAST_1);
            s3.setRegion(usEast1);
            log.info("Downloading an object from the S3 bucket.");
            S3Object object = s3.getObject(new GetObjectRequest(bucketName, key));
            log.info("Content-Type: " + object.getObjectMetadata().getContentType());
            inputStream = object.getObjectContent();

            File dirForOrder = new File(dirPath);
            if (!dirForOrder.exists()) {
                dirForOrder.mkdir();
            }

            outputStream = new FileOutputStream(new File(dirPath + fileSeparator + fileName));
            IOUtils.copy(inputStream, outputStream);
            inputStream.close();
            outputStream.close();
        } catch (FileNotFoundException e) {
            log.error(e);
        } catch (IOException e) {
            log.error(e);
        }
    }

    public static void uploadFileToS3Bucket(String bucketName, String key, String dirPath,
                                            String fileName) {
        AmazonS3 s3 = new AmazonS3Client(credentials);
        Region usEast1 = Region.getRegion(Regions.US_EAST_1);
        s3.setRegion(usEast1);
        s3.putObject(new PutObjectRequest(bucketName, key, new File(dirPath + fileSeparator +
                                                                    fileName)));
        try {
            FileUtil.delete(new File(dirPath));
        } catch (IOException e) {
            log.error(e);
        }

    }

    public static void main(String[] args) {
        readFileFromS3cketBucket("bucketName",
                                 "s3Key",
                                 "localFileSystemPath",
                                 "destinationFileName.pdf");
    }
}

希望这会有所帮助。快乐的编码！

[1] http://docs.aws.amazon.com/AmazonS3/latest/dev/RetrievingObjectUsingJava.html [2] http://docs.aws.amazon.com/AmazonS3/latest/dev/UploadObjSingleOpJava.html

如何使用java以编程方式在Amazon S3存储桶之间传输文件？

1 个答案: