Question

我正在将自定义指标从StackDriver导出到PubSub，这又将触发一个函数。我需要提取tableId和datasetId字段，这些字段又将用于删除不符合企业标准的bigquery表。

到目前为止，我尝试将“数据”字段的名称更改为protoPayload，textPayload，并且收到多个错误。其中一些错误是“无效键”，“使用字节代替dict”以及主要是“未定义变量”。

KeyError: 'District'

我希望能够提取单个字段（tableId和datasetId），并在BQ客户端中使用这些字段删除有问题的表。

更新：已复制对问题的评论

import base64

def hello_pubsub(event, context):
    pubsub_message = base64.b64decode(event['protoPayload']).decode('utf-8')
    print(pubsub_message)

Answer 1

使用此框架作为起点：

def hello_pubsub(event, context):
    if 'data' in event:
        pubsub_message = base64.b64decode(event['data']).decode('utf-8')
        print(pubsub_message)

        // Your custom data

        tableId = pubsub_message['requestMetadata']['serviceData']['tableInsertRequest']['resource']['tableName']['tableId']

        datasetId = pubsub_message['requestMetadata']['serviceData']['tableInsertRequest']['resource']['tableName']['datasetId']

从下面的注释中解码JSON（缺少底部）：

MY-MESSAGE: {
    "insertId":"-cr1b1ae1x199",
    "logName":" logs/cloudaudit.googleapis.com%2Factivity",
    "protoPayload":{
        "@type":"type.googleapis.com/google.cloud.audit.AuditLog",
        "authenticationInfo":{
        "principalEmail":"xxx"
    },
    "authorizationInfo":[{
        "granted":true,
        "permission":"bigquery.tables.create",
        "resource":"xxx"
    }],
    "methodName":"tableservice.insert",
    "requestMetadata":{
        "callerIp":"x",
        "callerSuppliedUserAgent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/x Safari/537.36,gzip(gfe)"},
        "resourceName":"x",
        serviceData"":{
            "@type":"type.googleapis.com/google.cloud.bigquery.logging.v1.AuditData",
            "tableInsertRequest":{
                "resource":{
                    "info":{},
                    "schemaJson":"{\n}",
                    "tableName":{
                        "datasetId":"citi_dev_gcp",
                        "projectId":"x",
                        "tableId":"outputCheck"
                    },
                    "view":{}
                }
            },
            "tableInsertResponse":{
                "resource":{
                    "createTime":"2019-06-20T18:42:21.106Z",
                    "expireTime":"2019-07-20T18:42:21.106Z",
                    "info":{},
                    "schemaJson":"{\n}",
                    "tableName":{
                        "datasetId":"name",
                        "projectId":"x",
                        "tableId":"outputCheck"
                    },
                    "updateTime":"2019-06-20T18:42:21.151Z",
                    "view":{}
                }
            }
        },
        "serviceName":"bigquery.googleapis.com","status":{}
    },
    "receiveTimestamp":

从pubsub / stackdriver消息中提取字段

1 个答案: