使用模板部署使用Data lake创建Azure HDInsight时出现问题

时间:2017-04-28 11:09:38

标签: azure azure-resource-manager hdinsight azure-data-lake

我正在尝试使用模板部署创建 Azure HDInsight with Data lake。但是我在执行模板时面临一个问题,因为我认为原因是“服务原则名称”与azure数据湖商店集成。

错误:

“message”:“DeploymentDocument'AmbariConfiguration_1_7'验证失败。错误:'访问datalake存储帐户时出错demodls:从AAD获取OAuth令牌时出错AppPrincipalId XXXXXX-XXXXXXXXX-XXXXX-XXX -XXXXX。

请在下面的屏幕截图中了解更多详情。

enter image description here

我尝试创建AD webapp并为应用分配“所有者”角色。然后我将其分配给Subscription的所有者。然后为应用添加了“Data Lake Permission”。但我仍然认为我可能会失踪。

群集集成代码段

"properties": {
                "clusterVersion": "[parameters('clusterVersion')]",
                "osType": "Linux",
                "tier": "standard",
                "clusterDefinition": {
                    "kind": "[parameters('clusterKind')]",
                    "configurations": {
                        "gateway": {
                            "restAuthCredential.isEnabled": true,
                            "restAuthCredential.username": "[parameters('clusterLoginUserName')]",
                            "restAuthCredential.password": "[parameters('clusterLoginPassword')]"
                        },
                        "core-site": {
                            "fs.defaultFS": "adl://home",
                            "dfs.adls.home.hostname": "demodls.azuredatalakestore.net",
                            "dfs.adls.home.mountpoint": "/clusters/democluster/"
                        },
                        "clusterIdentity": {
                            "clusterIdentity.applicationId": "XXXXX-XXXXX-XXXXX-XXXXX-XXXXX",
                            "clusterIdentity.certificate": "[parameters('identityCertificate')]",
                            "clusterIdentity.aadTenantId": "https://login.windows.net/XXXXXXXX-XXXX-XXXX-XXXXX-XXXXXXXXXX",
                            "clusterIdentity.resourceUri": "https://management.core.windows.net/",
                            "clusterIdentity.certificatePassword": "[parameters('identityCertificatePassword')]"
                        }
                    }
                },

在这里,我有一些疑问,比如

  1. 像“cluster.word”这样的“SecureString”值,“parameter.json”中的sshpassword应该以明文形式提供,还是必须将其转换为Securestring并为其提供安全字符串值?

  2. 字段“identityCertificate”应该是“base64”编码的“Certificate.pfx”文件内容,或者我必须将其转换为Base64 - > SecureString并在parameter.json中提供它?

  3. 非常感谢!感谢

    此致

2 个答案:

答案 0 :(得分:2)

identityCertificate应该是证书.pfx文件内容的base64编码字符串表示形式。它在ARM模板定义文件中标记为类型SecureString,以便在您获取部署历史记录时不会存储/返回明文。使用SecureString标记字段有助于确保密码和其他此类字段不会在您的部署历史记录中保留。

解决如何创建群集创建ARM模板的一种简单方法是转到Azure门户,并根据需要在模板中创建群集。在点击'创建'之前在总结'步骤,下载ARM模板以查看部署的内容。在'创建'旁边有一个链接。这样做。

我希望您注意到您如何指定主要ADLS帐户的不同之处。继续下载的ARM模板中的配置方式,你应该好好去。

答案 1 :(得分:0)

@Matt H

我在创建HDInsight时已经下载了在门户网站上生成的模板,即使它仍然无法正常工作。

请找到我的以下powershell脚本。

 //To Create Resources
 $resourceGroupName = "demoesprg"
 New-AzureRmResourceGroup -Name $resourceGroupName -Location "East US 2"
 $dataLakeStoreName = "demoespdls"
 New-AzureRmDataLakeStoreAccount -ResourceGroupName $resourceGroupName -Name $dataLakeStoreName -Location "East US 2"
 Test-AzureRmDataLakeStoreAccount -Name $dataLakeStoreName
 $myrootdir = "/"
 New-AzureRmDataLakeStoreItem -Folder -AccountName $dataLakeStoreName -Path $myrootdir/clusters/demoespcluster

 $templatefilepath = "C:\Azure-saml\template.json"
 $SSHpass = ConvertTo-SecureString -String "Demoesp1234$" -AsPlainText -Force

  //Create .pfx certificate
 $certFolder = "C:\Azure-saml\certs"
 $certFilePath = "$certFolder\demoespcert.pfx"
 $certStartDate = (Get-Date).Date
 $certStartDateStr = $certStartDate.ToString("MM/dd/yyyy")
 $certEndDate = $certStartDate.AddYears(1)
 $certEndDateStr = $certEndDate.ToString("MM/dd/yyyy")
 $certName = "demoespcert"
 $certPassword = "democert123$"
 $certPasswordSecureString = ConvertTo-SecureString $certPassword -AsPlainText -Force 
 $cert = New-SelfSignedCertificate -DnsName $certName -CertStoreLocation cert:\CurrentUser\My 
 $certThumbprint = $cert.Thumbprint
 $cert = (Get-ChildItem -Path cert:\CurrentUser\My\$certThumbprint) 
 Export-PfxCertificate -Cert $cert -FilePath $certFilePath -Password $certPasswordSecureString 
 $certificatePFX = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($certFilePath, $certPasswordSecureString)
 $credential = [System.Convert]::ToBase64String($certificatePFX.GetRawCertData())

 //create ActiceDriectory Application
 $application = New-AzureRmADApplication `
     -DisplayName "ESPSPN" `
     -HomePage "https://demoespcluster.hdinsight.net" `
     -IdentifierUris "https://demoespcluster.hdinsight.net" `
     -CertValue $credential  `
     -StartDate $certificatePFX.NotBefore  `
     -EndDate $certificatePFX.NotAfter 
 Start-Sleep -Seconds 20

 //Create Service Principla
 $applicationId = $application.ApplicationId
 $servicePrincipal = New-AzureRmADServicePrincipal -ApplicationId $applicationId
 $objectId = $servicePrincipal.Id

 //Assign Permissions
 Set-AzureRmDataLakeStoreItemAclEntry -AccountName $dataLakeStoreName -Path / -AceType User -Id $objectId -Permissions All
 Set-AzureRmDataLakeStoreItemAclEntry -AccountName $dataLakeStoreName -Path /clusters -AceType User -Id $objectId -Permissions All
 Set-AzureRmDataLakeStoreItemAclEntry -AccountName $dataLakeStoreName -Path /clusters/demoespcluster -AceType User -Id $objectId -Permissions All


 //Execute Scripts
 $tenantID = (Get-AzureRmContext).Tenant.TenantId
 $secureCert = [System.Convert]::ToBase64String((Get-Content $certFilePath -Encoding Byte))
 //$dsecureCert = ConvertTo-SecureString $secureCert -AsPlainText -Force

 New-AzureRmResourceGroupDeployment `
    -ResourceGroupName $resourceGroupName `
    -TemplateFile $templatefilepath `
    -identityCertificate $secureCert `
    -identityCertificatePassword $certPasswordSecureString `
    -clusterName  $certName `
    -clusterLoginPassword $SSHpass `
    -sshPassword $SSHpass `
    -servicePrincipalApplicationId $applicationId

错误:

  

New-AzureRmResourceGroupDeployment:11:15:00 PM - DeploymentDocument   'AmbariConfiguration_1_7'验证失败。错误:'错误时   访问datalake存储帐户dem​​oespdls:Access   拒绝。

我在这里缺少什么?

更新:脚本是正确的,但我的自签名证书有问题。一旦使用了有效的证书,我就能成功创建集群!感谢。