Question

在Azure数据工厂中，我需要重新运行4000多个失败的管道。可以在Azure门户用户界面中执行此操作，但是我尝试在Powershell中自动执行运行过程。

我无法在powershell中找到通过运行ID将失败的管道重新运行的命令/步骤。

Answer 1

我只是找到了一个教程Azure Data Factory: Detecting and Re-Running failed ADF Slices，它为您提供了Powershell脚本来自动执行失败的管道。

Powershell脚本：

Login-AzureRmAccount
$slices= @()
$tableName=@()
$failedSlices= @()
$failedSlicesCount= @()
$tableNames=@()

$Subscription="Provide Subscription ID"  

  Select-AzureRMSubscription -SubscriptionId  $Subscription    
$DataFactoryName="Provide Data Factory Name"
$resourceGroupName ="Porvide Resource Group Name for Data Factory"

$startDateTime ="2015-05-01" #Start Date for Slices
$endDateTime="2015-08-01" # End Date for Slices


#Get Dataset names in Data Factory - you can exlicitly give a table name using $tableName variable if you like to run only for an individual tablename
$tableNames = Get-AzureRMDataFactoryDataset -DataFactoryName $DataFactoryName -ResourceGroupName $resourceGroupName | ForEach {$_.DatasetName}

$tableNames #lists tablenames

foreach ($tableName in $tableNames)
{
    $slices += Get-AzureRMDataFactorySlice -DataFactoryName $DataFactoryName -DatasetName $tableName -StartDateTime $startDateTime -EndDateTime $endDateTime -ResourceGroupName $resourceGroupName -ErrorAction Stop
}


$failedSlices = $slices | Where {$_.Status -eq 'Failed'}

$failedSlicesCount = @($failedSlices).Count

if ( $failedSlicesCount -gt 0 ) 
{

    write-host "Total number of slices Failed:$failedSlicesCount"
    $Prompt = Read-host "Do you want to Rerun these failed slices? (Y | N)" 
    if ( $Prompt -eq "Y" -Or $Prompt -eq "y" )
    {

        foreach ($failed in $failedSlices)
        {
            write-host "Rerunning slice of Dataset "$($failed.DatasetName)" with StartDateTime "$($failed.Start)" and EndDateTime "$($failed.End)"" 
            Set-AzureRMDataFactorySliceStatus -UpdateType UpstreamInPipeline -Status Waiting -DataFactoryName $($failed.DataFactoryName) -DatasetName $($failed.DatasetName) -ResourceGroupName $resourceGroupName -StartDateTime "$($failed.Start)" -EndDateTime "$($failed.End)" 


        }
    }

}
else
{
    write-host "There are no Failed slices in the given time period."
}

希望这会有所帮助。

Answer 2

对于ADF V2，新的Az命令为

Invoke-AzDataFactoryV2Pipeline
      [-ResourceGroupName] <String>
      [-DataFactoryName] <String>
      [-PipelineName] <String>
      [[-ParameterFile] <String>]
      [-DefaultProfile <IAzureContextContainer>]
      [-WhatIf]
      [-Confirm]
      [<CommonParameters>]

source article

有更多变体

Answer 3

我在寻找相同问题的解决方案时发现了这个问题，所以这里是我所发现问题的答案。

如果您想重新运行整个管道，而又不关心Data Factory在技术上认为它是重新运行，则可以使用Get-AzDataFactoryV2PipelineRun cmdlet来获取运行列表，进行过滤失败的运行，然后在调用Invoke-AzureRmDataFactoryV2Pipeline

中使用相同的参数

在不久的将来，将更新cmdlet以允许真正的重新运行（基于对this issue在Microsoft Visually monitor Azure Data Factory文档上提出的某人的答复）。

如果您急于要进行真正的重新运行，则可以通过使用一些可选参数调用createRun来将功能包含在REST API中。

使用Powershell如何通过运行ID在Azure数据工厂中重新运行失败的管道？

3 个答案: