出于奇怪的原因并且仅在过去几天，Dataflow似乎无法发送步骤日志或更新UI步骤计数器。
我认为原因纯粹是IAM问题，但即使使用新的新项目，问题仍然存在。
值得注意的是，我的工作（自定义pubsub＆gt; bigquery管道）仍在执行中。
这是在worker级别的Stackdriver中找到的有效负载/跟踪：
{
insertId: "6719480855170687722:477955:0:122647"
jsonPayload: {
exception: "com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found
{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "(2efb88076acae577): Failed to publish the result of the work update. Causes: (2efb88076acae982): Could not update work; user does not have write access to project: [...]",
"reason" : "notFound"
} ],
"message" : "(2efb88076acae577): Failed to publish the result of the work update. Causes: (2efb88076acae982): Could not update work; user does not have write access to project: [...]",
"status" : "NOT_FOUND"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.dataflow.worker.runners.worker.DataflowWorkUnitClient.reportWorkItemStatus(DataflowWorkUnitClient.java:183)
at com.google.cloud.dataflow.worker.runners.worker.StreamingDataflowWorker.sendWorkerUpdatesToDataflowService(StreamingDataflowWorker.java:1163)
at com.google.cloud.dataflow.worker.runners.worker.StreamingDataflowWorker.reportPeriodicWorkerUpdates(StreamingDataflowWorker.java:1122)
at com.google.cloud.dataflow.worker.runners.worker.StreamingDataflowWorker$5.run(StreamingDataflowWorker.java:450)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
"
job: "2017-10-01_09_46_35-1778703197096293569"
logger: "com.google.cloud.dataflow.worker.runners.worker.StreamingDataflowWorker"
message: "Failed to send periodic counter updates"
thread: "33"
work: "<nil>"
worker: "ingest-paywall-seen-10010946-b8e5-harness-bnt1"
}
labels: {
compute.googleapis.com/resource_id: "6719480855170687722"
compute.googleapis.com/resource_name: "ingest-paywall-seen-10010946-b8e5-harness-bnt1"
compute.googleapis.com/resource_type: "instance"
dataflow.googleapis.com/job_id: "2017-10-01_09_46_35-1778703197096293569"
dataflow.googleapis.com/job_name: "ingest-paywall-seen"
dataflow.googleapis.com/region: "us-central1"
}
logName: "projects/[...]/logs/dataflow.googleapis.com%2Fworker"
receiveTimestamp: "2017-10-01T16:51:43.887618561Z"
resource: {
labels: {
job_id: "2017-10-01_09_46_35-1778703197096293569"
job_name: "ingest-paywall-seen"
project_id: "[...]"
region: "us-central1"
step_id: ""
}
type: "dataflow_step"
}
severity: "WARNING"
timestamp: "2017-10-01T16:51:24.024Z"
}
我显然检查了IAM以确保一切正常，并且在启用Dataflow API时，我的@dataflow-service-producer-prod.iam.gserviceaccount.com标识似乎具有默认角色。
有什么想法吗？