Question

我正在尝试在发布/订阅主题的情况下触发一个云函数来创建一个新的dataproc集群，该主题会在任何dataproc集群创建失败且状态码为零（status_code！= 0）的情况下获取消息）记录在云日志（dataproc活动日志）中。

我已经为上述情况编写了python代码，但是此云功能在由pub / sub触发后立即崩溃。

我的代码有什么问题，需要根据上述情况进行修改才能成功执行。

import base64
import json

import googleapiclient.discovery
from google.cloud import dataproc_v1 as dataproc

def dataproc_workflow(event, context):
    """
    Triggered by a Cloud Pub/Sub message containing a Dataproc status code != 0
    audit activity Stackdriver log message
    """
    pubsub_message = base64.b64decode(event['data']).decode('utf-8')
    msg_json = json.loads(pubsub_message)
    proto_payload = msg_json['protoPayload']
    resource_name = proto_payload['resourceName']
    email = proto_payload['authenticationInfo']['principalEmail']
    client = dataproc.ClusterControllerClient()
    create_cluster(client, project_id, zone, region, cluster_name)
    print(f"Cluster created: {cluster_name}.")

def create_cluster(client, project_id, zone, region, cluster_name):
    print('Creating cluster...')
    cluster_data = {
        'project_id': *********************,
        'cluster_name': simple,
        'region' : ******************,
        'config': {
            'config_bucket': ************,
    'gce_cluster_config': {
      'zone_uri': ********************,
      'subnetwork_uri': ********************,
      'internal_ip_only': true,
      'service_account_scopes': [
        'https://www.googleapis.com/auth/cloud-platform'
      ],
      'tags': [
        'dataproc-rule',
              ]
    },
            'master_config': {
                'num_instances': 1,
                'machine_type_uri': 'n1-standard-1'
            },
            'worker_config': {
                'num_instances': 2,
                'machine_type_uri': 'n1-standard-1'
                      }
        }     
    }
    response = client.create_cluster(project_id, region, cluster_data)
    result = response.result()
    print("After cluster create")
    
    return result

Answer 1

从代码片段讲起，似乎所有Dataproc群集配置都是硬编码的？例如project_id，region等，而不是从方法参数中读取。同样，create_cluster(client, project_id, zone, region, cluster_name)中传递的参数似乎没有定义在任何地方，只要触发工作流程，这肯定会导致程序崩溃。

如果这不是问题，并且有意忽略了这些变量，请附加完整的堆栈跟踪。

触发后云功能立即崩溃

1 个答案: