在最终成功之前,尝试使用Azure Pod Identity失败了很多次

时间:2019-04-01 08:44:10

标签: azure-aks

简介:我正试图让Azure Pod Identity在我们的群集中工作,以从KeyVault中读取机密,并且目前为止大部分都是成功的(到目前为止很好)。暂时,我们有两个密钥库,两个AzureIdentity,两个AzureIdentityBinding和两个Pod,它们各自使用它们的密钥库。

在测试时,两个Pod均相等-唯一不同的是它们的aadpodidbinding和一个环境变量,该环境变量指示要使用的密钥库。在启动时,吊舱连接到KeyVault,读取两个值并用Console.WriteLine打印它们。如果连接失败,则pod将会崩溃,并且k8s将重新启动它。

问题:一个Pod可能启动后可以立即从Keyvault读取,而另一个Pod则崩溃并重新启动-似乎是这样-持续5次才能获得访问令牌。

失败时,将引发以下异常:

Unhandled Exception: Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProviderException: Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/******************. Exception Message: Tried the following 3 methods to get an access token, but none of them worked.
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/******************. Exception Message: Tried to get token using Managed Service Identity. Access token could not be acquired. MSI ResponseCode: Forbidden, Response: no AzureAssignedIdentity found for pod:default/kv-test-be

Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/******************. Exception Message: Tried to get token using Visual Studio. Access token could not be acquired. Environment variable LOCALAPPDATA not set.
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/******************. Exception Message: Tried to get token using Azure CLI. Access token could not be acquired. No such file or directory

   at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAuthResultAsyncImpl(String authority, String resource, String scope)
   at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.<get_KeyVaultTokenCallback>b__8_0(String authority, String resource, String scope)
   at Microsoft.Azure.KeyVault.KeyVaultCredential.PostAuthenticate(HttpResponseMessage response)
   at Microsoft.Azure.KeyVault.KeyVaultCredential.ProcessHttpRequestAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Azure.KeyVault.KeyVaultClient.GetSecretsWithHttpMessagesAsync(String vaultBaseUrl, Nullable`1 maxresults, Dictionary`2 customHeaders, CancellationToken cancellationToken)
   at Microsoft.Azure.KeyVault.KeyVaultClientExtensions.GetSecretsAsync(IKeyVaultClient operations, String vaultBaseUrl, Nullable`1 maxresults, CancellationToken cancellationToken)
   at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.LoadAsync()
   at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.Load()
   at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
   at KeyvaultTest.Program.Main(String[] args) in /app/src/Program.cs:line 16

使用FlexVolume时,行为类似(最终我们的一组吊舱将在生产中使用),但是我发现使用两个相等的吊舱更容易与错误相关。

在等待Pod成功的同时,我在麦克风的日志中看到“绑定已删除”和“绑定已应用”消息。

我的问题:

  • 此行为是否“如预期的那样”并且可能记录在某处?
  • 是否可以应用设置以使“删除-应用”周期更快?
  • 还有什么可以做的事情来缩短Pod创建和应用身份绑定之间的时间?这个问题可能与https://github.com/Azure/aad-pod-identity/issues/145
  • 有关吗

源代码: Program.cs

using System;
using System.IO;
using System.Threading;
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;

namespace KeyvaultTest
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Starting Keyvault read");

            var configuration = new ConfigurationBuilder()
                .AddAzureKeyVault()
                .Build();

            var test1 = configuration.GetValue<string>("jtest");
            Console.WriteLine(test1);
            var test2 = configuration.GetValue<string>("jtest:jtest");

            Console.WriteLine(test2);
            Console.WriteLine("Finished Keyvault read");
        }
    }
}

KeyVaultConfiguration.cs.cs

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Text;
using System.Threading;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Azure.KeyVault;
using Microsoft.Azure.Services.AppAuthentication;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Configuration.AzureKeyVault;

namespace KeyvaultTest
{
    public static class KeyVaultConfiguration
    {
        public static IConfigurationBuilder AddAzureKeyVault(this IConfigurationBuilder builder)
        {
            var builtConfig = builder.Build();
            var keyVaultName = Environment.GetEnvironmentVariable("KV_NAME");

            if (string.IsNullOrWhiteSpace(keyVaultName))
            {
                throw new Exception("KV_NAME is not defined");
            }

            Console.WriteLine($"Using KV_NAME = {keyVaultName}");

            var azureServiceTokenProvider = new AzureServiceTokenProvider();
            var keyVaultClient = new KeyVaultClient(
                new KeyVaultClient.AuthenticationCallback(
                    azureServiceTokenProvider.KeyVaultTokenCallback));

            builder.AddAzureKeyVault(
                $"https://{keyVaultName}.vault.azure.net/",
                keyVaultClient,
                new DefaultKeyVaultSecretManager());

            return builder;
        }
    }
}

任何帮助,提示或想法都会受到赞赏。

注意:我已经将相同的问题发布到项目github页面https://github.com/Azure/aad-pod-identity/issues/181上的发布板上

1 个答案:

答案 0 :(得分:0)

我们面临着同样的问题。我们通过升级AAD Pod Identity解决了此问题。我们的版本是1.5,而将其升级到1.7解决了我们的问题。

在此之前,我们还已经将应用程序使用的软件包(Microsoft.Azure.Services.AppAuthentication和Azure.Security.KeyVault.Secrets)升级到了最新版本,但还不够。