Question

我在使用EC2角色假设方法在Flink中进行身份验证时遇到问题。这些工作正在通过aws emr add-steps提交。

以下代码在main方法中有效，并允许我查看流的详细信息。

val k = new AmazonKinesisClient(new InstanceProfileCredentialsProvider())
k.setRegion(Region.getRegion(Regions.fromName("my-region")))
println("Stream: " + k.describeStream("stream-name"))

但是，当尝试使用相同的方法在同一个EMR集群上进行身份验证时，我收到以下错误。

val config = new Properties()
config.put(AWSConfigConstants.AWS_REGION, "my-region")
config.put(AWSConfigConstants.AWS_CREDENTIALS_PROVIDER, "INSTANCE")
config.put(ConsumerConfigConstants.STREAM_INITIAL_POSITION, "LATEST")

val consumer = new FlinkKinesisConsumer[String](
  "stream-name", new SimpleStringSchema, config)

错误

Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: http://169.254.169.254/latest/meta-data/iam/security-credentials/
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
    at org.apache.flink.kinesis.shaded.com.amazonaws.internal.EC2MetadataClient.readResponse(EC2MetadataClient.java:111)

我还尝试更改AmazonKinesisClient.createKinesisClient方法以强制使用InstanceProfileCredentialsProvider。但不幸的是，这给出了同样的403错误。

public static AmazonKinesisClient createKinesisClient(Properties configProps) {
    AmazonKinesisClient client = new AmazonKinesisClient(new InstanceProfileCredentialsProvider());
    client.setRegion(Region.getRegion(Regions.fromName("my-region")));
    return client;
}

再次出错。

Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: http://169.254.169.254/latest/meta-data/iam/security-credentials/
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
    at org.apache.flink.kinesis.shaded.com.amazonaws.internal.EC2MetadataClient.readResponse(EC2MetadataClient.java:111)

Answer 1

如果您需要代理来访问外部资源，请确保添加-Dhttp.nonProxyHosts 169.254.169.254以允许服务器访问元数据存储。

HTTP响应代码：403在Flink中尝试AWS身份验证时

1 个答案: