Kubernetes云日志记录身份验证问题

时间:2019-11-14 19:25:11

标签: kubernetes google-kubernetes-engine stackdriver google-cloud-stackdriver google-cloud-logging

我一直在尝试切换我的应用程序以使用Google Cloud日志记录(又名Stackdriver)。我建立了一个使用Google Java Cloud Logging Library的自定义Logback附加程序。

当我在IDE中运行应用程序时,它会正确记录到Google Cloud Logging中。在GKE上的Kubernetes容器中运行它时,出现身份验证错误。但是,在两种情况下,我都使用相同的GCP服务帐户。

我甚至花了很多时间从Google凭据对象输出密钥,以证明它们在每种环境中都是相同的。他们是。

有人对我可能缺少的东西有任何想法吗?是否存在任何与上下文相关的设置,这些设置可能导致身份验证从容器内部失败?任何人都知道下一步该怎么做吗?


更新:这是堆栈跟踪:

java.lang.RuntimeException:java.util.concurrent.ExecutionException:com.google.cloud.logging.LoggingException:io.grpc.StatusRuntimeException:UNAUTHENTICATED:凭据应使用fail()而不是在java.lang上抛出异常。 RuntimeException:java.util.concurrent.ExecutionException:com.google.cloud.logging.LoggingException:io.grpc.StatusRuntimeException:UNAUTHENTICATED:凭据应使用fail()而不是在com.google.cloud.logging.LoggingImpl上引发异常。在uk.co.processflows.platform.util.StackdriverAppender.append(StackdriverAppender.kt:135)上的com.google.cloud.logging.LoggingImpl.write(LoggingImpl.java:560)上的flush(LoggingImpl.java:579)在ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:84)在ch.qos.logback处的uk.co.processflows.platform.util.StackdriverAppender.ktn(18)处。 ch.qos.log上的core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:51) back.classic.Logger.appendLoopOnAppenders(Logger.java:270)位于ch.qos.logback.classic.Logger.callAppenders(Logger.java:257)位于ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger。 java:421)at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:383)at ch.qos.logback.classic.Logger.error(Logger.java:538)at uk.co. uk.co.processflows.platform.push.PushMessagingConnectionHandlerImpl.broadcast(PushMessagingConnectionHandlerImpl.kt:257)的uk.co.processflows.platform.push的processflows.platform.pushnotification.PushNotificationClient.forward(PushNotificationClient.kt:58)位于java.base / jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(本地方法)处的PushMessagingConnectionHandlerImpl.broadcastMessageToTenant(PushMessagingConnectionHandlerImpl.kt:153)在java.base / jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Native:62处) ),位于java.base / jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodA ccessorImpl.java:43),位于java.base / java.lang.reflect.Method.invoke(Method.java:567),位于com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:50),位于com.sun.proxy。$ Proxy69.broadcastMessageToTenant(未知源),网址为uk.co.processflows.platform.workflow.TaskLockControllerImpl $ lockTask $ 2.run(TaskLockControllerImpl.kt:94),网址为java.base / java.util.concurrent .java的java.base / java.util.concurrent.FutureTask.run的.Executors $ RunnableAdapter.call(Executors.java:515),java.base / java.util.concurrent.ThreadPoolExecutor.runWorker的(FutureTask.java:264) (ThreadPoolExecutor.java:1128)at java.base / java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:628)at java.base / java.lang.Thread.run(Thread.java:835) )的原因:java.util.concurrent.ExecutionException:com.google.cloud.logging.LoggingException:io.grpc.StatusRuntimeException:UNAUTHENTICATED:凭据应使用fail()而不是在com.google上抛出异常.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:552)位于com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:431)位于com.google.common.util.concurrent .com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:68)上的.AbstractFuture $ TrustedFuture.get(AbstractFuture.java:97)在com.google.cloud.logging.LoggingImpl.flush(LoggingImpl .java:577)...省略了25个常见框架造成原因:com.google.cloud.logging.LoggingException:io.grpc.StatusRuntimeException:UNAUTHENTICATED:凭据应使用fail()而不是在com.google上抛出异常。请访问com.google.cloud.log.spi.v2.GrpcLoggingRpc $ 2.apply(GrpcLoggingRpc.java:184)上的cloud.logging.spi.v2.GrpcLoggingRpc $ 2.apply(GrpcLoggingRpc.java:190)。 api.core.ApiFutures $ GaxFunctionToGuavaFunction.apply(ApiFutures.java:204)位于com.google.common.util.concurrent.AbstractCatchingFuture $ CatchingFuture.doFallback(AbstractCatchingFuture.java:206)在com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:107)在com.google.common.util.concurrent.AbstractCatchingFuture $ CatchingFuture.doFallback(AbstractCatchingFuture.java:194)在com.google。 .common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30),位于com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1138),位于com.google.common.util.concurrent网址为com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:748)的.AbstractFuture.complete(AbstractFuture.java:958),网址为com.google.api.core.AbstractApiFuture $ InternalSettableFuture.setException(AbstractApiFuture) .java:95),位于com.google.api.gax.rpc.BatchedFuture.setException(BatchedFuture.java:55),位于com.google.api.gax.rpc.BatchedFuture.setException(BatchedFuture.java:55),位于com.google.api.core.AbstractApiFuture.setException(AbstractApiFuture.java:77) .google.api.gax.rpc.BatchedRequestIssuer.sendResult(BatchedRequestIssuer.java:84),网址为com.google.api.gax.rpc.BatchExecutor $ 1.onFailure (BatchExecutor.java:98),位于com.google.api.core.ApiFutures $ 1.onFailure(ApiFutures.java:68),位于com.google.common.util.concurrent.Futures $ CallbackListener.run(Futures.java: 1056),网址为com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1138),网址为com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)。网址为com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:748)的google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:958),网址为com.google.api.gax。 recoming.BasicRetryingFuture.handleAttempt(BasicRetryingFuture.java:179)位于com.google.api.gax.retrying.CallbackChainRetryingFuture $ AttemptCompletionListener.handle(CallbackChainRetryingFuture.java:135)位于com.google.api。网址为com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)处的AttemptCompletionListener.run(CallbackChainRetryingFuture.java:117) com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:958)位于com.google.common处的com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1138)。 com.google.api.core.AbstractApiFuture $ InternalSettableFuture.setException(AbstractApiFuture.java:95)处的util.concurrent.AbstractFuture.setException(AbstractFuture.java:748)在com.google.api.core.AbstractApiFuture.setException( com.google.api.gax.grpc.GrpcExceptionCallable $ ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)处的com.google.api.core.ApiFutures $ 1.onFailure(ApiFutures.java:68)的AbstractApiFuture.java:77) ),网址为com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30),网址为com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30),网址为com.google.common.util.concurrent.Futures $ CallbackListener.run(Futures.java:1056) .google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1138),位于com.google.common.util.concurrent.AbstractFuture.complete(AbstractF) uture.java:958),网址为com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:748),网址为io.grpc.stub.ClientCalls $ GrpcFuture.setException(ClientCalls.java:515),网址: io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)处的io.grpc.stub.ClientCalls $ UnaryStreamToFuture.onClose(ClientCalls.java:490)在io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)处在io.grpc.ForwardingClientCallListener $ SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)在io.grpc.internal.CensusStatsModule $ StatsClientInterceptor $ 1 $ 1.onClose(CensusStatsModule.java:699)在io.grpc.PartialonwardClient。位于io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)处的PartialForwardingClientCallListener.java:39)位于io.grpc.ForwardingClientCallListener $ SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:4)处0)在io.grpc.internal.CensusTracingModule $ TracingClientInterceptor $ 1 $ 1.onClose(CensusTracingModule.java:397)在io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:459)在io.grpc.internal。位于io.grpc.internal.ClientCallImpl $ ClientStreamListenerImpl.close(ClientCallImpl.java:546)处的ClientCallImpl.access $ 300(ClientCallImpl.java:63)位于io.grpc.internal.ClientCallImpl $ ClientStreamListenerImpl.access $ 600(ClientCallImpl.java: 467)位于io.grpc.internal.ClientCallImpl $ ClientStreamListenerImpl $ 1StreamClosed.runInContext(ClientCallImpl.java:584)位于io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)位于io.grpc.internal。网址为java.base / java.util.concurrent.Executors $ RunnableAdapter.call(Executors.java:515)处的SerializingExecutor.run(SerializingExecutor.java:123),位于java.base / java.util.concurrent.FutureTask.run( FutureTask.java:264),位于java.base / java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:3) 04)... ...省略了3个常见框架造成原因:com.google.api.gax.rpc.UnauthenticatedException:io.grpc.StatusRuntimeException:UNAUTHENTICATED:凭据应使用fail()而不是在com.google.api上抛出异常com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)处的com.google.api.gax.grpc.GrpcApiExceptionFactory处的.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:73) .create(GrpcApiExceptionFactory.java:60)at ...省略了30个共同的帧原因:io.grpc.StatusRuntimeException:未授权:凭据应使用fail()而不是在io.grpc.Status.asRuntimeException(Status)处引发异常。 java:532)at ... ...省略了22个常见框架造成原因:java.lang.NoSuchMethodError:io.grpc.MethodDescriptor.getServiceName()Ljava / lang / String;于io.grpc.CallCredentials2:app.RequestCeta:2app上的io.grpc.auth.GoogleAuthLibraryCallCredentials.applyRequestMetadata(GoogleAuthLibraryCallCredentials.java:103)处的io.grpc.auth.GoogleAuthLibraryCallCredentials.java:162处。 58)在io.grpc.internal.ForwardingConnectionClientClientTransport.newStream(ForwardingConnectionClientTransport.java:49)的io.grpc.internal.CallCredentialsApplyingTransportFactory $ CallCredentialsApplyingTransport.newStream(CallCredentialsApplyingTransportFactory.java:107)在io.grpc.in位于io.grpc.internal.ClientCallImpl.start(ClientCallImpl.java:245)处的io.grpc.internal.CensusTracingModule $ TracingClientInterceptor $ 1.start(CensusTracingModule.java:392)处的CallTracingTransport.newStream(InternalSubchannel.java:690)在io.grpc.ForwardingClientCall.start(ForwardingCl)处的io.grpc.internal.CensusStatsModule $ StatsClientInterceptor $ 1.start(CensusStatsModule.java:694)在com.google.api.gax.grpc.GrpcHeaderInterceptor $ 1.start(GrpcHeaderInterceptor.java:94)在io.grpc.stub.ClientCalls.startCall(ClientCalls.java:310)在ientCall.java:32)位于io.grpc.stub.ClientCalls.futureUnaryCall(ClientCalls.java:191)的com.google.api.gax.grpc.GrpcDirectCallable.futureCall(GrpcDirectCallable)的.grpc.stub.ClientCalls.asyncUnaryRequestCall(ClientCalls.java:282) .java:58),网址为com.google.api.gax.grpc.GrpcExceptionCallable.futureCall(GrpcExceptionCallable.java:64),网址为com.google.api.gax.rpc.AttemptCallable.call(AttemptCallable.java:86),网址为在com.google.api.gax.rpc.RetryingCallable.futureCall(RetryingCallable.java:41)在com.google.api.gax.rpc.RetryingCallable.futureCall(RetryingCallable.java:63)在com.google.api com.google.api.gax.rpc.UnaryCallable $ 1.futureCall(UnaryCallable.java:126)的com.google.api.gax.rpc的.gax.tracing.TracedBatchingCallable.futureCall(TracedBatchingCallable.java:82)。 UnaryCallable.futureCall(UnaryCallable.java:87)位于com.google.api.gax.rpc.BatchExecutor.processBatch(BatchExecutor.java:53)上的com.google.api.gax.rpc.BatchExecutor.processBatch(BatchExecutor.java:82)位于com.google.api。在com.google.api.gax.batching.ThresholdBatcher $ 1.run(ThresholdBatcher.java:76)处的gax.batching.ThresholdBatcher.pushCurrentBatch(ThresholdBatcher.java:233)在...上省略了6个常见框架


更新2:

我刚刚发现由堆栈跟踪层次结构引起的内部深处

原因:java.lang.NoSuchMethodError:io.grpc.MethodDescriptor.getServiceName()Ljava / lang / String;在io.grpc.auth.GoogleAuthLibraryCallCredentials.serviceUri(GoogleAuthLibraryCallCredentials.java:162)

听起来不像是身份验证问题。

3 个答案:

答案 0 :(得分:2)

更新最初回答此问题时,我没有看到堆栈跟踪,这使我相信您的问题根本与身份验证无关。我的答案可能与那些看到GKE中的GCP API身份验证问题的人有关,所以我将原来的内容留在这里。

原始如果遇到身份验证错误,则可能是您未正确使用服务帐户密钥。您可以阅读此tutorial,以了解应如何进行。简而言之,您需要将服务帐户密钥导出为JSON文件并将其作为秘密安装。

答案 1 :(得分:0)

鉴于使用该库的问题,您可能希望直接编写日志。关于创建结构化日志的doc可能会有所帮助。

答案 2 :(得分:0)

事实证明,GRPC jar存在依赖项兼容性问题。我们的应用程序使用GRPC的方式与Google Could Logging库一样。将我们的应用程序更改为使用针对Google Cloud日志记录库构建的GRPC的较早版本。

在容器中运行时,将以与本地运行不同的顺序加载jar文件。因此,为什么只在Kubernetes内部发生这种情况。