如何使用具有AWS临时凭证的Java从s3存储桶下载文件

时间:2018-05-16 06:49:38

标签: java amazon-web-services amazon-s3 aws-lambda

我有这个代码从s3存储桶下载文件到lambda服务器的tmp空间。

String url = "https://slack-automation.s3.amazonaws.com/slack.xlsx";

URL link = new URL(url);

/* //AmazonS3 s3Client = new AmazonS3Client(new ProfileCredentialsProvider());
   AmazonS3 s3Client = AmazonS3ClientBuilder.standard().build() ;
   S3Object object = s3Client.getObject(new GetObjectRequest(bucketName, key));
   InputStream objectData = object.getObjectContent();
   // Process the objectData stream.
   objectData.close();
 */

// Code to download
InputStream in = new BufferedInputStream(link.openStream());
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int n = 0;

while (-1 != (n = in.read(buf))) {
    out.write(buf, 0, n);
}

out.close();
in.close();
byte[] response = out.toByteArray();

FileOutputStream fos = new FileOutputStream("/tmp/" + fileName);
fos.write(response);
fos.close();
file = new File("/tmp/" + fileName);

//file = new File(filePath);
inputStream = new FileInputStream(file);
workBook = new XSSFWorkbook(inputStream);
workBookSheet = workBook.getSheet(workBook.getSheetName(0));
rowCount = workBookSheet.getPhysicalNumberOfRows();
formatter = new DataFormatter();

但问题是要访问此文件,必须提供公共权限。

如何通过获取此处提供的临时凭证将此文件下载到lambda服务器tmp空间:https://docs.aws.amazon.com/AmazonS3/latest/dev/AuthUsingTempSessionTokenJava.html

我无法实现,有人可以帮助我使用临时凭证下载文件的代码,而不像上面提到的方法那样传递我的accesskey和secretkey吗?

感谢Akshay Sing,我不知道如何用代码发表评论,我在你的回答后尝试的代码就在这里

public void setupWorkbook(String filePath)抛出IOException {

    String clientRegion = "eu-central-1";
    String roleARN = "myAwsArnRoleOverHere";
    String roleSessionName = "slackautomation";
    String bucketName = "slack-automation";
    String key = "slack.xlsx";


    String fileName = "slack.xlsx";



    try {

        AWSSecurityTokenService stsClient = AWSSecurityTokenServiceClientBuilder.standard()
                                                .withCredentials(new ProfileCredentialsProvider())
                                                .withRegion(clientRegion)
                                                .build();

        AssumeRoleRequest roleRequest = new AssumeRoleRequest()
                                                .withRoleArn(roleARN)
                                                .withRoleSessionName(roleSessionName);
        stsClient.assumeRole(roleRequest);

        // Start a session.
        GetSessionTokenRequest getSessionTokenRequest = new GetSessionTokenRequest();
        // The duration can be set to more than 3600 seconds only if temporary
        // credentials are requested by an IAM user rather than an account owner.
        getSessionTokenRequest.setDurationSeconds(900);
        GetSessionTokenResult sessionTokenResult = stsClient.getSessionToken(getSessionTokenRequest);
        Credentials sessionCredentials = sessionTokenResult.getCredentials();

        // Package the temporary security credentials as a BasicSessionCredentials object 
        // for an Amazon S3 client object to use.
        BasicSessionCredentials basicSessionCredentials = new BasicSessionCredentials(
                sessionCredentials.getAccessKeyId(), sessionCredentials.getSecretAccessKey(),
                sessionCredentials.getSessionToken());

        // Provide temporary security credentials so that the Amazon S3 client 
        // can send authenticated requests to Amazon S3. You create the client 
        // using the basicSessionCredentials object.
        AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
                                .withCredentials(new AWSStaticCredentialsProvider(basicSessionCredentials))
                                .withRegion(clientRegion)
                                .build();

        // Verify that assuming the role worked and the permissions are set correctly
        // by getting a set of object keys from the bucket.
        ObjectListing objects = s3Client.listObjects(bucketName);

        System.out.println("No. of Objects: " + objects.getObjectSummaries().size());
        S3Object fileObject = s3Client.getObject(new GetObjectRequest(bucketName, key));         
        InputStream in = new BufferedInputStream(fileObject.getObjectContent());

        ByteArrayOutputStream out = new ByteArrayOutputStream();

        byte[] buf = new byte[1024];

        int n = 0;

        while (-1 != (n = in.read(buf))) {

            out.write(buf, 0, n);

        }

        out.close();
        in.close();
        byte[] response = out.toByteArray();
        FileOutputStream fos = new FileOutputStream("/tmp/" + fileName);

        fos.write(response);

        fos.close();
        file = new File("/tmp/" + fileName);

        //file = new File(filePath);

        inputStream = new FileInputStream(file);

        workBook = new XSSFWorkbook(inputStream);

        workBookSheet = workBook.getSheet(workBook.getSheetName(0));

        rowCount = workBookSheet.getPhysicalNumberOfRows();

        formatter = new DataFormatter();


    }
    catch(AmazonServiceException e) {
        // The call was transmitted successfully, but Amazon S3 couldn't process 
        // it, so it returned an error response.
        e.printStackTrace();
    }
    catch(SdkClientException e) {
        // Amazon S3 couldn't be contacted for a response, or the client
        // couldn't parse the response from Amazon S3.
        e.printStackTrace();
    }

但它没有工作,它在“stsClient.assumeRole(roleRequest);”

上抛出错误

错误,只是行号没有多少线索。

2 个答案:

答案 0 :(得分:1)

上述问题的解决方案,我在这里写完整个代码

public void setupWorkbook(String filePath) throws IOException {



   // String bucketName = "slack-automation";
    String key = "slack.xlsx";


    String fileName = "slack.xlsx";


    String bucketName = System.getenv("bucket_name");

    try {



        AmazonS3 s3Client = new AmazonS3Client();

        ObjectListing objects = s3Client.listObjects(bucketName);

        System.out.println("No. of Objects: " + objects.getObjectSummaries().size());
        S3Object fileObject = s3Client.getObject(new GetObjectRequest(bucketName, key));         
        InputStream in = new BufferedInputStream(fileObject.getObjectContent());

        ByteArrayOutputStream out = new ByteArrayOutputStream();

        byte[] buf = new byte[1024];

        int n = 0;

        while (-1 != (n = in.read(buf))) {

            out.write(buf, 0, n);

        }

        out.close();
        in.close();
        byte[] response = out.toByteArray();
        FileOutputStream fos = new FileOutputStream("/tmp/" + fileName);

        fos.write(response);

        fos.close();
        file = new File("/tmp/" + fileName);

        //file = new File(filePath);

        inputStream = new FileInputStream(file);

        workBook = new XSSFWorkbook(inputStream);

        workBookSheet = workBook.getSheet(workBook.getSheetName(0));

        rowCount = workBookSheet.getPhysicalNumberOfRows();

        formatter = new DataFormatter();


    }
    catch(AmazonServiceException e) {
        // The call was transmitted successfully, but Amazon S3 couldn't process 
        // it, so it returned an error response.
        e.printStackTrace();
        System.out.println("FIRST EXCEPTION"+e.getMessage());
    }
    catch(SdkClientException e) {
        // Amazon S3 couldn't be contacted for a response, or the client
        // couldn't parse the response from Amazon S3.
        e.printStackTrace();
        System.out.println("SECOND EXCEPTION"+e.getMessage());
    }

只是这部分代码需要添加

   AmazonS3 s3Client = new AmazonS3Client();

    ObjectListing objects = s3Client.listObjects(bucketName);

    System.out.println("No. of Objects: " + objects.getObjectSummaries().size());
    S3Object fileObject = s3Client.getObject(new GetObjectRequest(bucketName, key));         
    InputStream in = new BufferedInputStream(fileObject.getObjectContent());

“AmazonS3 s3Client = new AmazonS3Client()”这一行将采用IAM角色,并且在不传递我的accesskey和secretkey的情况下将获取临时凭证,因为代码是从AWS Lambda运行的。就是这样,谢谢大家的帮助

答案 1 :(得分:0)

我目前正在使用以下程序从我的S3存储桶下载图像并存储它们以向用户返回图像的zip。我将为您排除zip创建代码:

要将文件下载到临时服务器,您必须首先获得所需S3Object的访问权限。在您的情况下,slack.xlsx存储在slack-automation桶中。

您提到要使用IAM角色提供给您的临时凭证。明智的做法是不在代码中披露凭据,同时确保提供给服务的凭据将自动刷新。您需要初始化s3Client 像这样:

AmazonS3Client s3Client = new AmazonS3Client(new InstanceProfileCredentialsProvider());

InstanceProfileCredentialsProvider从Amazon EC2实例元数据服务加载凭据(基于您的IAM角色)。

注意:确保正在使用的IAM角色实际上具有对存储桶的读访问权。

要获取S3Object,请使用:

S3Object fileObject = s3Client.getObject(new GetObjectRequest(bucketName, key));

在您的情况下,bucketName将是松弛自动化,密钥将是S3存储桶上的文件的名称(此处为slack.xlsx)。

然后,要将文件存储在您的服务器上,请使用copyInputStreamToFile

中的org.apache.commons.io.FileUtils功能
FileUtils.copyInputStreamToFile(s3Object.getObjectContent(), new File(fileName));