Question

我的Azure订阅上有两个单独的数据工厂，我们称它们为 DF-A 和另一个 DF-B 在Data Factory DF-A 中，我有一个管道，完成后，我希望运行 DF-B 上的管道；我将如何实现？

谢谢

Answer 1

在可能的情况下，它比在同一Azure数据工厂中执行另一个管道的管道要复杂得多。

在DF-A中，创建一个名为ExecuteExternalPipeline的管道，将以下JSON复制到“代码”选项卡中：

{
    "name": "ExecuteExternalPipeline",
    "properties": {
        "description": "Executes an ADF pipeline in a different ADF",
        "activities": [
            {
                "name": "StartPipelineThenWait",
                "description": "Calls the ADF REST API to start a pipeline in another ADF running using the MSI of this current ADF. Then it waits on a webhook callback",
                "type": "WebHook",
                "dependsOn": [],
                "userProperties": [],
                "typeProperties": {
                    "url": {
                        "value": "@concat(\n 'https://management.azure.com/subscriptions/',\n pipeline().parameters.SubscriptionID,\n '/resourceGroups/',pipeline().parameters.ResourceGroup,\n '/providers/Microsoft.DataFactory/factories/',\n pipeline().parameters.DataFactory,\n '/pipelines/',\n pipeline().parameters.Pipeline,\n '/createRun?api-version=2018-06-01'\n)",
                        "type": "Expression"
                    },
                    "method": "POST",
                    "body": {
                        "value": "@json(\n concat(\n  '{\n   \"InputFileName\": \"', pipeline().parameters.InputFileName, '\"\n  }'\n )\n)\n",
                        "type": "Expression"
                    },
                    "timeout": "20:00:00",
                    "authentication": {
                        "type": "MSI",
                        "resource": "https://management.azure.com"
                    }
                }
            },
            {
                "name": "ThrowErrorIfFailure",
                "type": "IfCondition",
                "dependsOn": [
                    {
                        "activity": "StartPipelineThenWait",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "userProperties": [],
                "typeProperties": {
                    "expression": {
                        "value": "@if(equals(activity('StartPipelineThenWait').status,'success'),true,json('throw an error!'))",
                        "type": "Expression"
                    }
                }
            }
        ],
        "parameters": {
            "SubscriptionID": {
                "type": "string",
                "defaultValue": "12345abcd-468e-472a-9761-9da416b14c0d"
            },
            "ResourceGroup": {
                "type": "string",
                "defaultValue": "DF-B-RG"
            },
            "DataFactory": {
                "type": "string",
                "defaultValue": "DF-B"
            },
            "Pipeline": {
                "type": "string",
                "defaultValue": "ChildPipeline"
            },
            "InputFileName": {
                "type": "string",
                "defaultValue": "File1.txt"
            }
        },
        "annotations": []
    }
}

然后使用以下代码在DF-B中创建ChildPipeline：

{
    "name": "ChildPipeline",
    "properties": {
        "activities": [
            {
                "name": "DoYourLogicHere",
                "description": "",
                "type": "WebActivity",
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "url": {
                        "value": "https://google.com",
                        "type": "Expression"
                    },
                    "method": "GET"
                }
            },
            {
                "name": "CallbackSuccess",
                "description": "Do not remove this activity. It notifies the process which executed this pipeline that the pipeline is complete.",
                "type": "WebActivity",
                "dependsOn": [
                    {
                        "activity": "DoYourLogicHere",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "url": {
                        "value": "@pipeline().parameters.callBackUri",
                        "type": "Expression"
                    },
                    "method": "POST",
                    "body": {
                        "value": "@json(concat('{\"status\": \"success\", \"pipelineRunId\": \"',pipeline().RunId,'\"}'))",
                        "type": "Expression"
                    }
                }
            },
            {
                "name": "CallbackFail",
                "description": "Do not remove this activity. It notifies the process which executed this pipeline that the pipeline is complete.",
                "type": "WebActivity",
                "dependsOn": [
                    {
                        "activity": "DoYourLogicHere",
                        "dependencyConditions": [
                            "Failed"
                        ]
                    }
                ],
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "url": {
                        "value": "@pipeline().parameters.callBackUri",
                        "type": "Expression"
                    },
                    "method": "POST",
                    "body": {
                        "value": "@json(concat('{\"status\": \"failure\", \"pipelineRunId\": \"',pipeline().RunId,'\"}'))",
                        "type": "Expression"
                    }
                }
            }
        ],
        "parameters": {
            "callBackUri": {
                "type": "string",
                "defaultValue": "https://google.com"
            },
            "InputFileName": {
                "type": "string",
                "defaultValue": "File1.txt"
            }
        },
        "annotations": []
    }
}

将DoYourLogicHere活动替换为您自己的活动，但保留两个回调活动。

然后，您需要找到DF-A的MSI（请参见Azure门户中DF-A的“属性”选项卡），并使其成为DF-B上的数据工厂贡献者，以便它可以在另一个中执行管道ADF。

Answer 2

在Logic应用程序设计器中，您可以创建两个管道运行步骤来触发不同Data Factory运行中的两个管道。
使用逻辑应用程序可以更轻松地实现这一目标。

创建重复发生触发器以安排执行，并创建两个 Azure Data Factory 操作以触发管道运行。

在 Azure Data Factory 操作中，选择创建管道运行操作。

摘要在这里：

当另一个管道在另一个数据工厂上完成时运行一个管道

2 个答案: