在azure FAILED上部署cloudera,JSON / ARM模板函数'copyIndex'错误

时间:2018-12-16 09:29:27

标签: json azure cloudera

我正在尝试使用下面的链接https://github.com/Azure/azure-quickstart-templates/tree/master/cloudera-on-centos在azure上安装cloudera,这给了我错误 “消息”:“部署模板验证失败:'493'行和'9'列上的模板资源'主节点'无效:在此位置不应使用模板函数'copyIndex'。该函数只能可以在指定了副本的资源中使用。有关用法的详细信息,请参见https://aka.ms/arm-copy。.有关用法的详细信息,请参见https://aka.ms/arm-template-expressions

代码看起来像这样

        "name": "master-node",
            "type": "Microsoft.Resources/deployments",
            "apiVersion": "2018-11-01",
            "dependsOn": [
                "Microsoft.Resources/deployments/shared-resources"
            ],
            "properties": {
                "mode": "Incremental",
                "template": {
                    "uri": "[concat(variables('scriptsUri'), '/master-node.json')]",
                    "contentVersion": "1.0.0.1",
                    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
                    "parameters": {
    "vnetID": {
      "type": "string"
    },
    "resourceAPIVersion": {
      "type": "string"
    },
    "dnsNamePrefix": {
      "type": "string",
      "defaultValue": "azuredevcluster"
    },
    "scriptsUri": {
      "type": "string"
    },
    "storageAccount": {
      "type": "object"
    },
    "vmCount": {
      "type": "int"
    },
    "vmSpec": {
      "type": "secureObject"
    },
    "networkSpec": {
      "type": "object"
    },
    "clusterSpec": {
      "type": "object"
    },
    "location": {
      "type": "string",
      "defaultValue": "[resourceGroup().location]",
      "metadata": {
        "description": "Location for all resources."
      }
    }
  },
                      "variables": {
    "singleQuote": "'",
    "masterIP": "[parameters('networkSpec').masterIP]",
    "workerIP": "[parameters('networkSpec').workerIP]",
    "vmName": "[concat(parameters('dnsNamePrefix'), '-mn')]",
    "nicName": "[concat(variables('vmName'), '-nic')]",
    "publicIPAddressName": "[concat(variables('vmName'), '-publicIP')]",
    "storageAccountName": "[parameters('storageAccount').prefix]",
    "securityGroupName": "[concat(parameters('dnsNamePrefix'), '-mn-sg')]"
  },
                    "resources": [
    {
      "type": "Microsoft.Network/networkSecurityGroups",
      "name": "[variables('securityGroupName')]",
      "location": "[parameters('location')]",
      "properties": {
        "securityRules": [
          {
            "name": "SSH",
            "properties": {
              "description": "Allows SSH traffic",
              "protocol": "Tcp",
              "sourcePortRange": "*",
              "destinationPortRange": "22",
              "sourceAddressPrefix": "*",
              "destinationAddressPrefix": "*",
              "access": "Allow",
              "priority": 100,
              "direction": "Inbound"
            }
          }
        ]
      }
    },
    {
      "type": "Microsoft.Network/publicIPAddresses",
      "name": "[concat(variables('publicIPAddressName'), copyIndex())]",
      "location": "[parameters('location')]",
      "copy": {
        "name": "publicIPLoop",
        "count": "[parameters('vmCount')]"
      },
      "properties": {
        "publicIPAllocationMethod": "Dynamic",
        "dnsSettings": {
          "domainNameLabel": "[concat(parameters('dnsNamePrefix'), '-mn', copyIndex())]"
        }
      }
    },
    {
      "type": "Microsoft.Network/networkInterfaces",
      "name": "[concat(variables('nicName'), copyIndex())]",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'), copyIndex())]",
        "[concat('Microsoft.Network/networkSecurityGroups/', variables('securityGroupName'))]"
      ],
      "copy": {
        "name": "masterNodeNicLoop",
        "count": "[parameters('vmCount')]"
      },
      "properties": {
        "ipConfigurations": [
          {
            "name": "ipconfig1",
            "properties": {
              "privateIPAllocationMethod": "Static",
              "privateIPAddress": "[concat(parameters('networkSpec').ipOctet01, add(parameters('networkSpec').ipOctet2, div(copyIndex(parameters('networkSpec').ipOctet3), 256)), '.', mod(copyIndex(parameters('networkSpec').ipOctet3), 256))]",
              "subnet": {
                "id": "[concat(parameters('vnetID'), '/subnets/', parameters('networkSpec').virtualNetworkSubnetName)]"
              },
              "publicIPAddress": {
                "id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(variables('publicIPAddressName'), copyIndex()))]"
              }
            }
          }
        ],
        "networkSecurityGroup": {
          "id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('securityGroupName'))]"
        }
      }
    },
    {
      "type": "Microsoft.Compute/virtualMachines",
      "apiVersion": "2017-03-30",
      "name": "[concat(variables('vmName'), copyIndex())]",
      "plan": "[parameters('vmSpec').imageInfo.plan]",
      "location": "[parameters('location')]",
      "copy":{
        "name": "masterNodeNicLoop",
        "count": "[parameters('vmCount')]"
      },
      "dependsOn": [
        "[concat('Microsoft.Network/networkInterfaces/', variables('nicName'), copyIndex())]"
      ],
      "properties": {
        "availabilitySet": {
          "id": "[resourceId('Microsoft.Compute/availabilitySets', parameters('vmSpec').masterNodeASName)]"
        },
        "hardwareProfile": {
          "vmSize": "[parameters('vmSpec').vmSize]"
        },
        "osProfile": {
          "computerName": "[reference(concat(variables('publicIPAddressName'), copyIndex())).dnsSettings.fqdn]",
          "adminUsername": "[parameters('vmSpec').adminUsername]",
          "adminPassword": "[parameters('vmSpec').adminPassword]",
          "linuxConfiguration": {
            "disablePasswordAuthentication": false
          }
        },
        "storageProfile": {
          "imageReference": "[parameters('vmSpec').imageInfo.imageReference]",
          "osDisk": {
            "name":"[concat(variables('vmName'), copyIndex(),'_OSDisk')]",
            "caching": "ReadWrite",
            "createOption": "FromImage"
          },
          "dataDisks": [
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk1')]",
              "diskSizeGB": 512,
              "lun": 0,
              "createOption": "Empty"
            },
            {
              "name":  "[concat(variables('vmName'), copyIndex(),'_DataDisk2')]",
              "diskSizeGB": 512,
              "lun": 1,
              "createOption": "Empty"
            },
            {
              "name":  "[concat(variables('vmName'), copyIndex(),'_DataDisk3')]",
              "diskSizeGB": 512,
              "lun": 2,
              "createOption": "Empty"
            },
            {
              "name":  "[concat(variables('vmName'), copyIndex(),'_DataDisk4')]",
              "diskSizeGB": 512,
              "lun": 3,
              "createOption": "Empty"
            }
          ]
        },
        "networkProfile": {
          "networkInterfaces": [
            {
              "id": "[resourceId('Microsoft.Network/networkInterfaces',concat(variables('nicName'), copyIndex()))]"
            }
          ]
        }
      }
    },
    {
      "type": "Microsoft.Compute/virtualMachines/extensions",
      "name": "[concat(variables('vmName'), copyIndex(), '/prepareDisks')]",
      "location": "[parameters('location')]",
      "copy": {
        "name": "nameNodeNicLoop",
        "count": "[parameters('vmCount')]"
      },
      "dependsOn": [
        "[concat('Microsoft.Compute/virtualMachines/', variables('vmName'), copyIndex())]"
      ],
      "properties": {
        "publisher": "Microsoft.Azure.Extensions",
        "type": "CustomScript",
        "typeHandlerVersion": "2.0",
        "autoUpgradeMinorVersion": true,
        "settings": {
          "fileUris": [
            "[concat(parameters('scriptsUri'), '/scripts/prepare-masternode-disks.sh')]",
            "[concat(parameters('scriptsUri'), '/scripts/initialize-node.sh')]"
          ],
          "commandToExecute": "[concat('sh initialize-node.sh ', variables('singleQuote'), variables('masterIP'), variables('singleQuote'), ' ', variables('singleQuote'), variables('workerIP'), variables('singleQuote'), ' ', variables('singleQuote'), parameters('dnsNamePrefix'), variables('singleQuote'), ' ', variables('singleQuote'), reference(concat(variables('publicIPAddressName'), copyIndex())).dnsSettings.fqdn, variables('singleQuote'), ' ', variables('singleQuote'), parameters('clusterSpec').masterNodeCount, variables('singleQuote'), ' ', variables('singleQuote'), parameters('clusterSpec').dataNodeCount, variables('singleQuote'), ' ', variables('singleQuote'), parameters('vmSpec').adminUsername, variables('singleQuote'), ' ', 'masternode', ' >> /var/log/cloudera-azure-initialize.log 2>&1')]"
        }
      }
    }
  ],
  "outputs": {
    "fqdn": {
      "type": "string",
      "value": "[reference(concat(variables('publicIPAddressName'), '0')).dnsSettings.fqdn]"
    }
  }
                },
                "parameters": {
                    "vnetID": {
                        "value": "[variables('VNetId')]"
                    },
                    "resourceAPIVersion": {
                        "value": "[variables('resourceAPIVersion')]"
                    },
                    "dnsNamePrefix": {
                        "value": "[parameters('dnsNamePrefix')]"
                    },
                    "scriptsUri": {
                        "value": "[variables('scriptsUri')]"
                    },
                    "storageAccount": {
                        "value": "[variables('masterStorageAccount')]"
                    },
                    "vmCount": {
                        "value": "[variables('clusterSpec').masterNodeCount]"
                    },
                    "vmSpec": {
                        "value": "[variables('vmSpec')]"
                    },
                    "networkSpec": {
                        "value": "[variables('networkSpec')]"
                    },
                    "clusterSpec": {
                        "value": "[variables('clusterSpec')]"
                    }
                }
            }
        },
        {
            "name": "data-node",
            "type": "Microsoft.Resources/deployments",
            "apiVersion": "2018-11-01",
            "dependsOn": [
                "Microsoft.Resources/deployments/shared-resources"
            ],
            "properties": {
                "mode": "Incremental",
                "template": {
                    "uri": "[variables('data-nodeUri')]",
                    "contentVersion": "1.0.0.1",
                    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
                    "parameters": {
    "vnetID": {
      "type": "string"
    },
    "dnsNamePrefix": {
      "type": "string",
      "defaultValue": "azuredevcluster"
    },
    "scriptsUri": {
      "type": "string"
    },
    "vmCount": {
      "type": "int"
    },
    "vmSpec": {
      "type": "secureObject"
    },
    "networkSpec": {
      "type": "object"
    },
    "clusterSpec": {
      "type": "object"
    },
    "location": {
      "type": "string",
      "defaultValue": "[resourceGroup().location]",
      "metadata": {
        "description": "Location for all resources."
      }
    }
  },
                     "variables": {
    "singleQuote": "'",
    "masterIP": "[parameters('networkSpec').masterIP]",
    "workerIP": "[parameters('networkSpec').workerIP]",
    "vmName": "[concat(parameters('dnsNamePrefix'), '-dn')]",
    "nicName": "[concat(variables('vmName'), '-nic')]",
    "publicIPAddressName": "[concat(variables('vmName'), '-publicIP')]",
    "securityGroupName": "[concat(parameters('dnsNamePrefix'), '-dn-sg')]"
  },
  "resources": [
    {
      "type": "Microsoft.Network/networkSecurityGroups",
      "name": "[variables('securityGroupName')]",
      "location": "[parameters('location')]",
      "properties": {
        "securityRules": [
          {
            "name": "SSH",
            "properties": {
              "description": "Allows SSH traffic",
              "protocol": "Tcp",
              "sourcePortRange": "*",
              "destinationPortRange": "22",
              "sourceAddressPrefix": "*",
              "destinationAddressPrefix": "*",
              "access": "Allow",
              "priority": 100,
              "direction": "Inbound"
            }
          }
        ]
      }
    },
    {
      "type": "Microsoft.Network/publicIPAddresses",
      "name": "[concat(variables('publicIPAddressName'), copyIndex())]",
      "location": "[parameters('location')]",
      "copy": {
        "name": "publicIPLoop",
        "count": "[parameters('vmCount')]"
      },
      "properties": {
        "publicIPAllocationMethod": "Dynamic",
        "dnsSettings": {
          "domainNameLabel": "[concat(parameters('dnsNamePrefix'), '-dn', copyIndex())]"
        }
      }
    },
    {
      "type": "Microsoft.Network/networkInterfaces",
      "name": "[concat(variables('nicName'), copyIndex())]",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'), copyIndex())]",
        "[concat('Microsoft.Network/networkSecurityGroups/', variables('securityGroupName'))]"
      ],
      "copy": {
        "name": "dataNodeNicLoop",
        "count": "[parameters('vmCount')]"
      },
      "properties": {
        "ipConfigurations": [
          {
            "name": "ipconfig1",
            "properties": {
              "privateIPAllocationMethod": "Static",
              "privateIPAddress": "[concat(parameters('networkSpec').ipOctet01, add(parameters('networkSpec').ipOctet2, div(copyIndex(parameters('networkSpec').datanodeIpOctet3), 256)), '.', mod(copyIndex(parameters('networkSpec').datanodeIpOctet3), 256))]",
              "subnet": {
                "id": "[concat(parameters('vnetID'), '/subnets/', parameters('networkSpec').virtualNetworkSubnetName)]"
              },
              "publicIPAddress": {
                "id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(variables('publicIPAddressName'), copyIndex()))]"
              }
            }
          }
        ],
        "networkSecurityGroup": {
          "id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('securityGroupName'))]"
        }
      }
    },
    {
      "type": "Microsoft.Compute/virtualMachines",
      "apiVersion": "2017-03-30",
      "name": "[concat(variables('vmName'), copyIndex())]",
      "plan": {
        "name": "cloudera-centos-6",
        "publisher": "cloudera",
        "product": "cloudera-centos-6"
      },
      "location": "[parameters('location')]",
      "copy": {
        "name": "dataNodeNicLoop",
        "count": "[parameters('vmCount')]"
      },
      "dependsOn": [
        "[concat('Microsoft.Network/networkInterfaces/', variables('nicName'), copyIndex())]"
      ],
      "properties": {
        "availabilitySet": {
          "id": "[resourceId('Microsoft.Compute/availabilitySets', parameters('vmSpec').dataNodeASName)]"
        },
        "hardwareProfile": {
          "vmSize": "[parameters('vmSpec').vmSize]"
        },
        "osProfile": {
          "computerName": "[reference(concat(variables('publicIPAddressName'), copyIndex())).dnsSettings.fqdn]",
          "adminUsername": "[parameters('vmSpec').adminUsername]",
          "adminPassword": "[parameters('vmSpec').adminPassword]",
          "linuxConfiguration": {
            "disablePasswordAuthentication": false
          }
        },
        "storageProfile": {
          "imageReference": {
            "publisher": "cloudera",
            "offer": "cloudera-centos-6",
            "sku": "cloudera-centos-6",
            "version": "latest"
          },
          "osDisk": {
            "name": "[concat(variables('vmName'), copyIndex(),'_OSDisk')]",
            "caching": "ReadWrite",
            "createOption": "FromImage"
          },
          "dataDisks": [
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk1')]",
              "diskSizeGB": 512,
              "lun": 0,
              "createOption": "Empty"
            },
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk2')]",
              "diskSizeGB": 1023,
              "lun": 1,
              "createOption": "Empty"
            },
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk3')]",
              "diskSizeGB": 1023,
              "lun": 2,
              "createOption": "Empty"
            },
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk4')]",
              "diskSizeGB": 1023,
              "lun": 3,
              "createOption": "Empty"
            },
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk5')]",
              "diskSizeGB": 1023,
              "lun": 4,
              "createOption": "Empty"
            },
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk6')]",
              "diskSizeGB": 1023,
              "lun": 5,
              "createOption": "Empty"
            }
          ]
        },
        "networkProfile": {
          "networkInterfaces": [
            {
              "id": "[resourceId('Microsoft.Network/networkInterfaces',concat(variables('nicName'), copyIndex()))]"
            }
          ]
        }
      }
    },
    {
      "type": "Microsoft.Compute/virtualMachines/extensions",
      "name": "[concat(variables('vmName'), copyIndex(), '/prepareDisks')]",
      "location": "[parameters('location')]",
      "copy": {
        "name": "dataNodeNicLoop",
        "count": "[parameters('vmCount')]"
      },
      "dependsOn": [
        "[concat('Microsoft.Compute/virtualMachines/', variables('vmName'), copyIndex())]"
      ],
      "properties": {
        "publisher": "Microsoft.Azure.Extensions",
        "type": "CustomScript",
        "typeHandlerVersion": "2.0",
        "autoUpgradeMinorVersion": true,
        "settings": {
          "fileUris": [
            "[concat(parameters('scriptsUri'), '/scripts/prepare-datanode-disks.sh')]",
            "[concat(parameters('scriptsUri'), '/scripts/initialize-node.sh')]"
          ],
          "commandToExecute": "[concat('sh initialize-node.sh ', variables('singleQuote'), variables('masterIP'), variables('singleQuote'), ' ', variables('singleQuote'), variables('workerIP'), variables('singleQuote'), ' ', variables('singleQuote'), parameters('dnsNamePrefix'), variables('singleQuote'), ' ', variables('singleQuote'), reference(concat(variables('publicIPAddressName'), copyIndex())).dnsSettings.fqdn, variables('singleQuote'), ' ', variables('singleQuote'), parameters('clusterSpec').masterNodeCount, variables('singleQuote'), ' ', variables('singleQuote'), parameters('clusterSpec').dataNodeCount, variables('singleQuote'), ' ', variables('singleQuote'), parameters('vmSpec').adminUsername, variables('singleQuote'), ' ', 'datanode', ' >> /var/log/cloudera-azure-initialize.log 2>&1')]"
        }
      }
    }
  ]
                },
                "parameters": {
                    "vnetID": {
                        "value": "[variables('VNetId')]"
                    },
                    "templateAPIVersion": {
                        "value": "[variables('templateAPIVersion')]"
                    },
                    "resourceAPIVersion": {
                        "value": "[variables('resourceAPIVersion')]"
                    },
                    "dnsNamePrefix": {
                        "value": "[parameters('dnsNamePrefix')]"
                    },
                    "scriptsUri": {
                        "value": "[variables('scriptsUri')]"
                    },
                    "storageAccount": {
                        "value": "[variables('workerStorageAccount')]"
                    },
                    "vmCount": {
                        "value": "[variables('clusterSpec').dataNodeCount]"
                    },
                    "vmSpec": {
                        "value": "[variables('vmSpec')]"
                    },
                    "networkSpec": {
                        "value": "[variables('networkSpec')]"
                    },
                    "clusterSpec": {
                        "value": "[variables('clusterSpec')]"
                    }
                }
            }
        },
        {
            "name": "setup-cloudera",
            "type": "Microsoft.Resources/deployments",
            "apiVersion": "2018-11-01",
            "dependsOn": [
                "Microsoft.Resources/deployments/data-node",
                "Microsoft.Resources/deployments/master-node"
            ],
            "properties": {
                "mode": "Incremental",
                "template": {
                    "uri": "[concat(variables('scriptsUri'), '/setup-cloudera.json')]",
                    "contentVersion": "1.0.0.1"
                },
                "parameters": {
                    "resourceAPIVersion": {
                        "value": "[variables('resourceAPIVersion')]"
                    },
                    "dnsNamePrefix": {
                        "value": "[parameters('dnsNamePrefix')]"
                    },
                    "scriptsUri": {
                        "value": "[variables('scriptsUri')]"
                    },
                    "vmSpec": {
                        "value": "[variables('vmSpec')]"
                    },
                    "networkSpec": {
                        "value": "[variables('networkSpec')]"
                    },
                    "clusterSpec": {
                        "value": "[variables('clusterSpec')]"
                    },
                    "fqdn": {
                        "value": "[reference('master-node').outputs.fqdn.value]"
                    },
                    "cmUsername": {
                        "value": "[parameters('cmUsername')]"
                    },
                    "cmPassword": {
                        "value": "[parameters('cmPassword')]"
                    },
                    "company": {
                        "value": "[parameters('Company')]"
                    },
                    "emailAddress": {
                        "value": "[parameters('emailAddress')]"
                    },
                    "businessPhone": {
                        "value": "[parameters('businessPhone')]"
                    },
                    "firstName": {
                        "value": "[parameters('firstName')]"
                    },
                    "lastName": {
                        "value": "[parameters('lastName')]"
                    },
                    "jobRole": {
                        "value": "[parameters('jobRole')]"
                    },
                    "jobFunction": {
                        "value": "[parameters('jobFunction')]"
                    },
                    "installCDH": {
                        "value": "[variables('installCDH')]"
                    }
                }
            }
        }
    ]
}

1 个答案:

答案 0 :(得分:0)

是的,我已经找到了问题所在(可悲的是,此后可能还会有更多问题)。问题来自您正在使用嵌套的INLINE模板以及它们如何(或不可以)与参考函数一起工作:

"[reference(concat(variables('publicIPAddressName'), copyIndex())).dnsSettings.fqdn]",

这是多数民众赞成在打破的部分。因此,我的建议是将其设为单个模板而不嵌套它们,或者将其转换为完整的嵌套模板(使用templateLink属性,而不是template属性:

{
    "name": "name",
    "type": "Microsoft.Resources/deployments",
    "apiVersion": "2017-05-10",
    "properties": {
        "mode": "Incremental",
        "templateLink": {
            "uri": "url_goes_here"
        },
        "parameters": {
            "prefix": {
                "value": "xxx"
            }
        }
    }
},

我建议您不要使用嵌套的INLINE模板。他们太前卫了。