在运行数据融合管道以将GCS的csv文件加载到BigQuery时,遇到了与数据进程取消配置有关的问题

时间:2019-05-07 16:55:25

标签: google-cloud-data-fusion

我正在使用数据融合来创建管道,该管道会将CSV数据从GCS加载到BigQuery。当我进行预览时,效果很好。但是当我部署管道时,它给了我下面的错误。

ERROR   io.cdap.cdap.internal.provision.task.ProvisioningTask#151-provisioning-service-13   DEPROVISION task failed in REQUESTING_DELETE state for program run program_run:default.gcstobqsample.-SNAPSHOT.workflow.DataPipelineWorkflow.31a8341b-70d6-11e9-9c94-92fdc3807015.
com.google.api.gax.rpc.FailedPreconditionException: io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Cannot delete cluster 'cdap-gcstobqsa-31a8341b-70d6-11e9-9c94-92fdc3807015' while it has other pending delete operations.
    at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:59) ~[na:na]
    at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72) ~[na:na]
    at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60) ~[na:na]
    at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:95) ~[na:na]
    at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:61) ~[na:na]
    at com.google.common.util.concurrent.Futures$4.run(Futures.java:1123) ~[com.google.guava.guava-13.0.1.jar:na]
    at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:435) ~[na:na]
    at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:900) ~[com.google.guava.guava-13.0.1.jar:na]
    at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:811) ~[com.google.guava.guava-13.0.1.jar:na]
    at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:675) ~[com.google.guava.guava-13.0.1.jar:na]
    at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:492) ~[na:na]
    at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:467) ~[na:na]
    at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41) ~[na:na]
    at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684) ~[na:na]
    at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41) ~[na:na]
    at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:392) ~[na:na]
    at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:475) ~[na:na]
    at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63) ~[na:na]
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:557) ~[na:na]
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:478) ~[na:na]
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:590) ~[na:na]
    at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[na:na]
    at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) ~[na:na]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_212]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_212]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_212]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_212]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_212]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_212]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_212]
Caused by: io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Cannot delete cluster 'cdap-gcstobqsa-31a8341b-70d6-11e9-9c94-92fdc3807015' while it has other pending delete operations.
    at io.grpc.Status.asRuntimeException(Status.java:526) ~[na:na]
    ... 19 common frames omitted

1 个答案:

答案 0 :(得分:0)

该错误似乎是在删除作为清理操作的dataproc集群时发生的。这不一定表示管道故障的原因。您能否更早地检查管道日志中的其他错误?附加完整的日志以进行管道运行非常好。