简化了无服务器AWS步骤功能的等待

时间:2019-12-13 10:48:47

标签: python-3.x serverless-framework aws-step-functions

我已经通过Serverless编写了AWS Step函数。逻辑如下所示

1. Execute Job
2. Get job status 
3. Check if job succeeded 
4. If not succeded wait for 60 seconds 

现在,我已经编写了如下定义的步进函数

...
stepFunctions: 
  stateMachines: 
    myStateMachine: 
       definition:
         states: 
           ...
           Execute Job: 
             ... 
           Get Job Status: 
             Type: Task 
             Next: Is Job Succeeded?
             # This job check if job is succeeded or not and sets a variable 
             # job_succeeded = true
           Is Job Succeeded?
             Type: Choice
             Choices:
             - Variable: "$.job_succeded"
               BooleanEquals: true
               Next: Next Lambda Function  
             Default: Wait for job
           Wait for job:
             Type: Wait
             Seconds: 60
             Next: Get Job Status
           Next Lambda Function: 
              ...
           ...

这非常有效!

但是问题是,我必须为异步执行的每个作业编写4个lambda,因此,如果要执行4个这样的任务,将有4 * 4步骤,即16个步骤。

这使得步进函数中看起来有很多lambda,并且很难看到实际的流程。

这4个功能可以概括为以下pseduo代码

job_succeded = false
while job is not complete:
    job_succeded = get_job_status() 
    if not job_succeded:
        sleep 60 seconds 

可能的解决方案是

stepFunctions: 
  stateMachines: 
    myStateMachine: 
       definition:
         states: 
           ...
           Execute Job: 
             ... 
           Get Job Status: 
             Type: RepeatWhile
             Condition:
             - Variable: "$.job_succeded"
               BooleanEquals: true
               Next: Next Lambda Function
             WaitPeriod: 60 seconds 
             # This job check if job is succeeded or not and sets a variable 
             # job_succeeded = true
           Next Lambda Function: 
              ...
           ...

我知道这不是当前情况,但这可以简化许多工作流程。 还有其他方法可以简化此过程吗?

1 个答案:

答案 0 :(得分:0)

欢迎使用步进功能!不幸的是,当他们写下来时,他们非常very肿。我不知道这是否有帮助,但是对于像这样的简单事情,我将使用一个lambda函数来完成所有任务。

您可以使用其他参数定义步进函数,以告诉函数要执行的操作

  Task_1:
    Type: Task
    Resource:
      Fn::GetAtt:
        - MyLambdaFunction
        - Arn
    Parameters:
      execute: task_1
    Next: Task_2

  Task_2:
    Type: Task
    Resource:
      Fn::GetAtt:
        - MyLambdaFunction
        - Arn
    Parameters:
      execute: task_2
    Next: Task_3 

在您的lambda函数中,只需设置一些简单的if语句:

  if (event.execute === 'task_1') {
    // run task 1 code
  }
  else if (event.execute === 'task_2') {
    // run task 2 code
  }

我不太确定您要在问题中做什么,但是您也可以从lambda输出变量,然后触发不同的步骤。您可以为此使用选择状态

      ChoiceState:
        Type: Choice
        Choices:
          # response from lambda should be {executing: 'task_1', status: 'running'}
          - Variable: "$.executing" 
            StringEquals: task_1
          - Variable: "$.status"
            StringEquals: running
          Next: Wait

        - And:
          - Variable: "$.executing"
            StringEquals: task_1
          - Variable: "$.status"
            StringEquals: done
          Next: Task_2

        - And:
          - Variable: "$.executing"
            StringEquals: task_2
          - Variable: "$.status"
            StringEquals: running
          Next: Wait

        - And:
          - Variable: "$.executing"
            StringEquals: task_2
          - Variable: "$.status"
            StringEquals: done
          Next: Completed

        Wait:
          Type: Wait
          Seconds: 10
          Next: CheckExecution

        Completed:
          Type: Succeed

      CheckExecution:
        Type: Task
        Resource:
          Fn::GetAtt:
            - MyLambdaFunction
            - Arn
        Next: ChoiceState

这不是完整的解决方案,因为在这里写下来太复杂了。但希望能给您一些其他想法,供您玩

此外,根据状态机的运行方式,此方法可在最大程度减少lambda冷启动的情况下提高性能。