使用Azure

时间:2019-07-08 12:28:41

标签: azure powershell automation ssas

我有一个使用Azure数据工厂在Azure SQL数据库上提取数据,处理和泵送数据的过程。最后,SSAS表格模型使用来自Azure SQL的数据。到目前为止,使用Azure Data Factory v2自动化将数据泵送到Azure SQL的整个过程都是自动化的,但是我无法找到一种自动处理SSAS表格模型的方法。请让我知道是否有办法做到这一点。

我希望ADF v2以某种方式触发它,以便表格在新数据被泵入SQL后立即获取更新的数据。由于我无法使用ADF v2进行此操作,因此尝试使用Powershell。我在这里面临的障碍是,我需要传递两个凭据来处理表格 1. Azure SQL的SQL Server身份验证 2.一名模拟用户的Windows凭据 我是Powershell的新手,在通过这两个凭据时遇到问题

Powershell脚本:-

param($ServerName="Your Server Name", $DBName="Your DB Name", $ProcessTypeDim="ProcessFull",$ProcessTypeMG="ProcessFull", 
$Transactional="Y", $Parallel="Y",$MaxParallel=2,$MaxCmdPerBatch=1, $PrintCmd="N",
$logFilePath="Your Path where you want to save the log file")
## Add the AMO namespace
Write-Output "Process Starts for database $DBName :" | Out-File -FilePath $logFilePath
$loadInfo = [Reflection.Assembly]::LoadWithPartialName("Microsoft.AnalysisServices")
if ($Transactional -eq "Y") {$TransactionalB=$true} else {$TransactionalB=$false}
if ($Parallel -eq "Y") {$ParallelB=$true} else {$ParallelB=$false}
$server = New-Object Microsoft.AnalysisServices.Server
$server.connect($ServerName)
if ($server.name -eq $null) {
 Write-Output ("Server '{0}' not found" -f $ServerName)
 Write-Output "Server '{0}' not found" -f $ServerName | Out-File -FilePath $logFilePath -Append
 break
}
$DB = $server.Databases.FindByName($DBName)
if ($DB -eq $null) {
 Write-Output ("Database '{0}' not found" -f $DBName)
 Write-Output "Database '{0}' not found" -f $DBName | Out-File -FilePath $logFilePath -Append
 break
}
$date_Start=Get-Date
Write-Output("Load start time {0}" -f (Get-Date -uformat "%H:%M:%S") )
Write-Output "Load start time: "  (Get-Date -uformat "%H:%M:%S") | Out-File -FilePath $logFilePath -Append

Write-Output("----------------------------------------------------------------")
Write-Output "----------------------------------------------------------------"| Out-File -FilePath $logFilePath -Append
Write-Output("Server  : {0}" -f $Server.Name)
Write-Output "Server  :"   $Server.Name | Out-File -FilePath $logFilePath -Append
Write-Output("Database: {0}" -f $DB.Name)
Write-Output "Database: "  $DB.Name | Out-File -FilePath $logFilePath -Append
Write-Output("DB State: {0}" -f $DB.State)
Write-Output "DB State: "  $DB.State | Out-File -FilePath $logFilePath -Append
Write-Output("DB Size : {0}MB" -f 

    ($DB.EstimatedSize/1024/1024).ToString("#,##0"))
    Write-Output "DB Size : "  ($DB.EstimatedSize/1024/1024).ToString("#,##0")| Out-File -FilePath $logFilePath -Append
    Write-Output("----------------------------------------------------------------")
    Write-Output "----------------------------------------------------------------"| Out-File -FilePath $logFilePath -Append
    Write-Output("DB processing started.   Time: {0}" -f (Get-Date -uformat "%H:%M:%S"))
    Write-Output "DB processing started.   Time: " (Get-Date -uformat "%H:%M:%S")| Out-File -FilePath $logFilePath -Append
    $server.CaptureXml=$TRUE # Just capture server statements, dont execute them
    #Process dimensions
    foreach ($dim in $DB.Dimensions) {
      $dim.Process($ProcessTypeDim)
    } # Dimensions
    #Process cubes
    foreach ($cube in $DB.Cubes) {
     foreach ($mg in $cube.MeasureGroups) {
      foreach ($part in $mg.Partitions) {
       $part.Process($ProcessTypeMG)
      }
     }
    }
    # Separate step to process all linked measure groups. Linke MG does not have partitions
    foreach ($cube in $DB.Cubes) {
     foreach ($mg in $cube.MeasureGroups) {
      if ($mg.IsLinked) {
       $mg.Process($ProcessTypeMG)
      }
     }
    }

    $server.CaptureXML = $FALSE # Finish capturing statements. All statements are in Server.CaptureLog
    $cmdBatch = @"
    <Batch xmlns="http://schemas.microsoft.com/analysisservices/2003/engine">
      <Parallel MaxParallel="##MaxParallel##">
    ##ProcessCmd##
      </Parallel>
    </Batch>
    "@
    $cmdBatch = $cmdBatch -replace("##MaxParallel##",$MaxParallel)
    #$ErrorActionPreference = "SilentlyContinue"
    $currentCmdNo=0; $currentCmdInBatchNo=0;
    $processCmd="";$currentBatchNo=0
    $TotalCmdCount = $Server.CaptureLog.Count

    foreach ($cmdLine in $Server.CaptureLog) {
     $currentCmdNo = $currentCmdNo + 1
     $processCmd = $processCmd + $cmdLine + "`n"
     $currentCmdInBatchNo=$currentCmdInBatchNo + 1
     if ($currentCmdInBatchNo -ge $MaxCmdPerBatch -or $currentCmdNo -eq $TotalCmdCount) { #MaxCmdPerBatch reached, execute commands
      $processCmd = $cmdBatch -replace("##ProcessCmd##", $processCmd) 
      if ($PrintCmd -eq "Y") { Write-Output($processCmd) }
      $currentBatchNo = $currentBatchNo + 1;
      Write-Output("=== Startining batch No {0}. Time: {1} ..." -f $currentBatchNo, (Get-Date -uformat "%H:%M:%S"))
      Write-Output "=== Startining batch No $currentBatchNo. Time: ..."  (Get-Date -uformat "%H:%M:%S")| Out-File -FilePath $logFilePath -Append
      $Result = $Server.Execute($processCmd)
      # Report errors and warnings
      foreach ($res in $Result) {
       foreach ($msg in $res.Messages) {
        if ($msg.Description -ne $null) {
                        Write-Output("{0}" -f $msg.Description)
                        Write-Output $msg.Description| Out-File -FilePath $logFilePath -Append
                        } 
       }
      }
      # Reset temp values
      $processCmd = ""; $currentCmdInBatchNo=0;
     }
    }#foreach 

    Write-Output("-------------------------Cube status----------------------------")
    Write-Output("-------------------------Cube status----------------------------")| Out-File -FilePath $logFilePath -Append

    $data=$DB.Cubes|select name,state,lastprocessed
    Write-Output($data)
    Write-Output $data| Out-File -FilePath $logFilePath -Append 

    Write-Output("--------------------Dimension status----------------------------")
    Write-Output("--------------------Dimension status----------------------------")| Out-File -FilePath $logFilePath -Append

    $data=$DB.Dimensions|select name,state,lastprocessed
    Write-Output($data)
    Write-Output $data| Out-File -FilePath $logFilePath -Append 

    Write-Output("-----------------Dimension related to cubes---------------------")
    Write-Output("-----------------Dimension related to cubes---------------------")| Out-File -FilePath $logFilePath -Append

    foreach ($cube in $DB.Cubes) {
        Write-Output("Cube Name: $cube")
        Write-Output "Cube Name: $cube"| Out-File -FilePath $logFilePath -Append
            foreach ($dim in $DB.Dimensions) {
            Write-Output("                         $dim")
            Write-Output "                         $dim"| Out-File -FilePath $logFilePath -Append

        } 
    }

    Write-Output("----------------------------------------------------------------")
    Write-Output("----------------------------------------------------------------")| Out-File -FilePath $logFilePath -Append 
    $date_End=Get-Date
    Write-Output("Load End Time: {0}" -f (Get-Date -uformat "%H:%M:%S"))
    Write-Output "Load End Time: "  (Get-Date -uformat "%H:%M:%S")| Out-File -FilePath $logFilePath -Append

    $ptime="Total Processing Time :"+($date_End-$date_Start).Hours+" Hours, "+($date_End-$date_Start).Minutes+" Mins, "+($date_End-$date_Start).Seconds+" Secs "
    Write-Output $ptime
    Write-Output $ptime | Out-File -FilePath $logFilePath -Append 

错误消息-

  

OLE DB或ODBC错误:登录超时已到期; HYT00;与网络有关   或建立实例时发生了特定于实例的错误   与SQL Server的连接。找不到服务器或无法访问服务器。校验   如果实例名称正确,并且SQL Server配置为允许   远程连接。有关更多信息,请参见SQL Server联机丛书。   08001;命名管道提供程序:无法打开与SQL Server的连接   [5]。 ; 08001。

1 个答案:

答案 0 :(得分:0)

为使答案对其他人可见,我总结了评论中共享的答案OP:

通过使用SSAS中的模拟服务帐户解决了该问题