Question

如何使用ARM模板和Ambari API获取Edge节点的私有IP？

我正在使用ARM模板的以下edgenode部分安装Edge节点。我想为我的自定义应用程序获取Private IP Edge节点。如何使用ARM模板或使用edgenodeName使用Ambari？

{
  'name': '[concat(parameters('clusterName'),'/', parameters('edgenodeName'))]',
  'type': 'Microsoft.HDInsight/clusters/applications',
  'apiVersion': '2015-03-01-preview',
    'dependsOn': [
        '[concat('Microsoft.HDInsight/clusters/', parameters('clusterName'))]'
    ],
    'properties': {
    'marketPlaceIdentifier': 'EmptyEdgeNode',
    'computeProfile': {
      'roles': [{
        'name': 'edgenode',
        'targetInstanceCount': 1,
        'hardwareProfile': {
          'vmSize': '[parameters('edgenodeSize')]'
        }
      }]
    },
    'installScriptActions': [],
    'uninstallScriptActions': [],
    'httpsEndpoints': [],
    'applicationType': 'CustomApplication'
  }
}

更新1： -

这是我在resources.azure.com

中的json表示

{
  "id": "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.HDInsight/clusters/$clusterName",
  "name": "$clusterName",
  "type": "Microsoft.HDInsight/clusters",
  "location": "Central US",
  "etag": "33908087-88d4-43e6-bad4-7668bb90fa39",
  "tags": null,
  "properties": {
    "clusterVersion": "3.5.1000.0",
    "osType": "Linux",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/spark-3.5.1000.0.9988582.json",
      "kind": "SPARK",
      "componentVersion": {
        "Spark": "1.6"
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "Standard_D12_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "$userName"
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Network/virtualNetworks/$vnetName",
            "subnet": "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Network/virtualNetworks/$vnetName/subnets/default"
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 1,
          "hardwareProfile": {
            "vmSize": "Standard_D12_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "$userName"
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Network/virtualNetworks/$vnetName",
            "subnet": "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Network/virtualNetworks/$vnetName/subnets/default"
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Medium"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "$userName"
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Network/virtualNetworks/$vnetName",
            "subnet": "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Network/virtualNetworks/$vnetName/subnets/default"
          }
        },
        {
          "name": "edgenode1",
          "targetInstanceCount": 1,
          "hardwareProfile": {
            "vmSize": "Standard_D3_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "$userName"
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Network/virtualNetworks/$vnetName",
            "subnet": "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Network/virtualNetworks/$vnetName/subnets/default"
          }
        }
      ]
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2017-04-26T07:44:54.4",
    "quotaInfo": {
      "coresUsed": 16
    },
    "connectivityEndpoints": [
      {
        "name": "SSH",
        "protocol": "TCP",
        "location": "$clusterName-ssh.azurehdinsight.net",
        "port": 22
      },
      {
        "name": "HTTPS",
        "protocol": "TCP",
        "location": "$clusterName.azurehdinsight.net",
        "port": 443
      }
    ],
    "tier": "standard"
  }
}

Answer 1

您可以使用Ambari API获取Edge节点IP。模板部署成功后，您可以使用以下脚本列出边缘节点IP。

#!/bin/bash
PASSWORD=$1
CLUSTERNAME=$2
###list all host private IP
for HOSTNAME in $(curl -u admin:$PASSWORD -sS -G "https://$CLUSTERNAME.azurehdinsight.net/api/v1/clusters/$CLUSTERNAME/hosts" | jq -r '.items[].Hosts.host_name')
do
    IP=$(curl -u admin:$PASSWORD -sS -G "https://$CLUSTERNAME.azurehdinsight.net/api/v1/clusters/$CLUSTERNAME/hosts/$HOSTNAME" | jq -r '.Hosts.ip')
  echo "$HOSTNAME <--> $IP" >>host.txt
done
cat host.txt |grep '^ed'|awk -F\> '{print $2 }'

在host.txt中，您将获得所有类似的主机IP。

ed11-******.gx.internal.cloudapp.net <--> 10.4.0.4
ed20-******.gx.internal.cloudapp.net <--> 10.4.0.8
hn0-******.gx.internal.cloudapp.net <--> 10.4.0.18
hn1-******.gx.internal.cloudapp.net <--> 10.4.0.13
wn0-******.gx.internal.cloudapp.net <--> 10.4.0.7
zk1-******.gx.internal.cloudapp.net <--> 10.4.0.12
zk3-******.gx.internal.cloudapp.net <--> 10.4.0.9
zk5-******.gx.internal.cloudapp.net <--> 10.4.0.10

您可以执行如下脚本：

[root@shui home]# ./deploy.sh <password> <clustername>
 10.4.0.4
 10.4.0.8

如何在Azure中使用ARM模板或Ambari API获取Edge节点的专用IP

1 个答案: