使用scala时,lambda的AWS凭据无法正常工作

时间:2017-12-12 00:23:53

标签: scala amazon-web-services amazon-s3 aws-lambda aws-sdk

尝试使用使用DefaultCredentialProvider提供的凭据时,AWS lambda函数不起作用。

我需要将凭据传递给S3才能运行。

代码

def initializeAwsCredentials():AWSCredentials = {
    var credentials: AWSCredentials  = null
    try {
      credentials = new ProfileCredentialsProvider().getCredentials
    } catch {
      case e: Exception => {
        throw new AmazonClientException(
          "Cannot load the credentials from the credential profiles file. " +
            "Please make sure that your credentials file is at the correct " +
            "location (~/.aws/credentials), and is in valid format.",
          e);
      }

    }
    return credentials
  }

 def buildS3API(credentials: AWSCredentials): AmazonS3 = {
  new AmazonS3Client(credentials)
}

// inside handle request
val credentials = initializeAwsCredentials()
println("Credetials have been retrieved successfully")

println("Build S3 API using the constructor provided")
val s3 = buildS3API(credentials)
s3.setRegion(region)
println("S3 API is now available")

错误

{
  "errorMessage": "Cannot load the credentials from the credential profiles file. Please make sure that your credentials file is at the correct location (~/.aws/credentials), and is in valid format.",
  "errorType": "com.amazonaws.AmazonClientException",
  "stackTrace": [
    "example.Main$.initializeAwsCredentials(Hello.scala:52)",
    "example.Main$.handleRequest(Hello.scala:125)",
    "example.Main.handleRequest(Hello.scala)",
    "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
    "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
    "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
    "java.lang.reflect.Method.invoke(Method.java:498)"
  ],
  "cause": {
    "errorMessage": "java.lang.NullPointerException",
    "errorType": "java.lang.NullPointerException",
    "stackTrace": [
      "com.amazonaws.auth.profile.ProfilesConfigFile.<init>(ProfilesConfigFile.java:143)",
      "com.amazonaws.auth.profile.ProfilesConfigFile.<init>(ProfilesConfigFile.java:132)",
      "com.amazonaws.auth.profile.ProfilesConfigFile.<init>(ProfilesConfigFile.java:99)",
      "com.amazonaws.auth.profile.ProfileCredentialsProvider.getCredentials(ProfileCredentialsProvider.java:135)",
      "example.Main$.initializeAwsCredentials(Hello.scala:45)",
      "example.Main$.handleRequest(Hello.scala:125)",
      "example.Main.handleRequest(Hello.scala)",
      "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
      "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
      "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
      "java.lang.reflect.Method.invoke(Method.java:498)"
    ]
  }
}

更新

使用 InstanceProfileCredentialsProvider 代替抛出错误:

val provider: InstanceProfileCredentialsProvider = new InstanceProfileCredentialsProvider()
credentials = provider.getCredentials()

给我错误:

"cause": {
    "errorMessage": "Unable to load credentials from Amazon EC2 metadata service",
    "errorType": "com.amazonaws.AmazonClientException",
    "stackTrace": [
      "com.amazonaws.auth.InstanceProfileCredentialsProvider.handleError(InstanceProfileCredentialsProvider.java:244)",
      "com.amazonaws.auth.InstanceProfileCredentialsProvider.loadCredentials(InstanceProfileCredentialsProvider.java:225)",
      "com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:124)",
      "example.Main$.initializeAwsCredentials(Hello.scala:46)",
      "example.Main$.handleRequest(Hello.scala:126)",
      "example.Main.handleRequest(Hello.scala)",
      "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
      "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
      "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
      "java.lang.reflect.Method.invoke(Method.java:498)"
    ],
    "cause": {
      "errorMessage": "Connection refused (Connection refused)",
      "errorType": "java.net.ConnectException",
      "stackTrace": [
        "java.net.PlainSocketImpl.socketConnect(Native Method)",
        "java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)",
        "java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)",
        "java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)",
        "java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)",
        "java.net.Socket.connect(Socket.java:589)",
        "sun.net.NetworkClient.doConnect(NetworkClient.java:175)",
        "sun.net.www.http.HttpClient.openServer(HttpClient.java:463)",
        "sun.net.www.http.HttpClient.openServer(HttpClient.java:558)",
        "sun.net.www.http.HttpClient.<init>(HttpClient.java:242)",
        "sun.net.www.http.HttpClient.New(HttpClient.java:339)",
        "sun.net.www.http.HttpClient.New(HttpClient.java:357)",
        "sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1202)",
        "sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138)",
        "sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032)",
        "sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:966)",
        "com.amazonaws.internal.EC2MetadataClient.readResource(EC2MetadataClient.java:90)",
        "com.amazonaws.internal.EC2MetadataClient.getDefaultCredentials(EC2MetadataClient.java:55)",
        "com.amazonaws.auth.InstanceProfileCredentialsProvider.loadCredentials(InstanceProfileCredentialsProvider.java:186)",
        "com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:124)",
        "example.Main$.initializeAwsCredentials(Hello.scala:46)",
        "example.Main$.handleRequest(Hello.scala:126)",
        "example.Main.handleRequest(Hello.scala)",
        "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
        "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
        "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
        "java.lang.reflect.Method.invoke(Method.java:498)"
      ]
    }
  }
}

使用lambda时将以下配置为环境变量也失败:

Lambda was unable to configure your environment variables because the 
environment variables you have provided contains reserved keys that are 
currently not supported for modification. Reserved keys used in this 
request: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY

4 个答案:

答案 0 :(得分:1)

对于Lambda函数,您需要使用IAM角色作为凭据。然后,您可以使用DefaultAWSCredentialsProviderChain或InstanceProfileCredentialsProvider从IAM角色中检索凭据。

Class InstanceProfileCredentialsProvider

以下是使用InstanceProfileCredentialsProvider的示例:

   AWSCredentialsProvider credentialsProvider = null;
    try {
        credentialsProvider = new InstanceProfileCredentialsProvider();
        // Verify we can fetch credentials
        credentialsProvider.getCredentials();
        System.out.println("Obtained credentials.");
    } catch (AmazonClientException e) {
        System.out.println("Unable to obtain credentials", e);
        return -1;
    }

    System.out.println("Using credentials with access key id: " + credentialsProvider.getCredentials().getAWSAccessKeyId());

答案 1 :(得分:1)

我不确定您是否需要显式凭据提供程序。在AWS Lambda内部,凭证可以通过lambda可以承担的角色自动提供。我知道我从来没有明确地做过。

http://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/loading-node-credentials-lambda.html

答案 2 :(得分:0)

我在Java中也有类似情况。希望这里可以采用相同的方法。我将本地切换到EnvironmentVariableCredentialsProvider,通过本地Java程序“运行配置”中的env选项卡提供了凭据。然后我将jar上传到Lambda函数,也有Env配置,只需在其中提供凭据即可。

答案 3 :(得分:0)

在Java的最新版本的AWS开发工具包(1.11.4xx)中,几乎所有服务都具有可用于快速创建客户端的“客户端构建器”。

val snsClient = AmazonSNSClientBuilder.defaultClient()
val s3Client = AmazonS3ClientBuilder.defaultClient()
val dynamoDbClient = AmazonDynamoDBAsyncClientBuilder.defaultClient()
val sesClient = AmazonSimpleEmailServiceAsyncClientBuilder.defaultClient()
// ...

在Lambda中,defaultClient()的工作原理非常好,因为它将创建一个使用适当提供程序的客户端。该提供程序将凭据与lambda执行角色中定义的权限一起使用。

在本地环境中,defaultClient也可以很好地工作,因为它可以获取主机凭据。之所以有效,是因为defaultClient用于寻找

default credentials provider chain
  1. Env Vars
  2. Java系统属性
  3. 凭据配置文件
  4. ECS容器凭据
  5. 实例配置文件凭据

此方法也很简洁,但是您也可以使用客户端构建器使用特定的凭据“设置/配置”来创建客户端。


适用于Java v2的AWS开发工具包

如果您要使用Java SDK的新版本(>=2.1),可以使用create方法来获取客户端(尽管我仅将其用于实验新的SDK)

val s3Client = S3AsyncClient.create()
val dynamoDbClient = DynamoDbAsyncClient.create()
// ...