Question

我正在尝试使用Terraform在Azure上配置Windows VM并同时引导它。我能想到的是terraform Provisioner。出于测试目的，我这样编写预配器：

provisioner "remote-exec" {
    inline = [
      "md c:/terraform",
    ]
    connection {
      type     = "winrm"
      host   = "${azurerm_network_interface.vmstamp.private_ip_address}"
      user     = "${var.admin_username}"
      password = "${var.admin_password}"
      https    = false
      insecure = true
      timeout = "1m"
    }
}

模板成功配置了VM，但在尝试将VM与WinRM连接时产生错误。

azurerm_virtual_machine.vmstamp: Still creating... (5m50s elapsed)
azurerm_virtual_machine.vmstamp (remote-exec): Connecting to remote host via WinRM...
azurerm_virtual_machine.vmstamp (remote-exec):   Host: 10.237.249.146
azurerm_virtual_machine.vmstamp (remote-exec):   Port: 5985
azurerm_virtual_machine.vmstamp (remote-exec):   User: azadmin
azurerm_virtual_machine.vmstamp (remote-exec):   Password: true
azurerm_virtual_machine.vmstamp (remote-exec):   HTTPS: true
azurerm_virtual_machine.vmstamp (remote-exec):   Insecure: true
azurerm_virtual_machine.vmstamp (remote-exec):   NTLM: false
azurerm_virtual_machine.vmstamp (remote-exec):   CACert: false
azurerm_virtual_machine.vmstamp: Still creating... (6m0s elapsed)

Error: Error applying plan:

1 error(s) occurred:

* azurerm_virtual_machine.vmstamp: timeout - last error: unknown error Post https://10.237.249.146:5985/wsman: dial tcp 10.237.249.146:5985: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

我尝试过的几件事：

1）正在考虑防火墙问题。但是打包程序可以使用以下代码在同一台笔记本电脑上启动连接到新建的Windows VM：

"communicator": "winrm",
"winrm_use_ssl": "true",
"winrm_insecure": "true",
"winrm_timeout": "3m",
"winrm_username": "packer",

2）尝试过https=true和https=false，都失败了。

3）尝试过use_ntlm=true和use_ntlm=false，都失败了。

4）尝试过port=5985和port=5986，都失败了。 5986实际上不是从市场映像中侦听新的Windows VM。

Terraform Provisioner的正确配置是什么？

Answer 1

关于您的问题，我进行了测试。您可以通过Packer查看Windows图像的结果：

您可以看到该Winrm也有超时。对于Terraform的另一项测试，我也为winrm超时。因此，我建议您可以使用vm扩展名启用winrm，确保打开了端口5985，然后使用预配器执行您想要的操作。

更新

也许有些遗漏，所以我没有像你一样成功。但是有一个过程应该注意，Packer设置证书的URL。文档here还显示了如何使用证书的URL启用winrm。所以我认为Packer只是自己做，但是Terraform不会为您做。

Answer 2

@Charles Xu，注释不能放置太多字符，我将输出和代码粘贴到此处。加壳程序脚本输出：

azadmin@AZLAPSLOG1001:~/packer/winrmopen$ /usr/local/packer/packer build --var-file=variables.json Windows2016.json
azure-arm output will be in this color.

==> azure-arm: Running builder ...
    azure-arm: Creating Azure Resource Manager (ARM) client ...
    azure-arm: You have provided Object_ID which is no longer needed, azure packer builder determines this dynamically from the authentication token
==> azure-arm: Using existing resource group ...
==> azure-arm:  -> ResourceGroupName : '###Resource Group Replacement####'
==> azure-arm:  -> Location          : 'canadacentral'
==> azure-arm: Validating deployment template ...
==> azure-arm:  -> ResourceGroupName : '###Resource Group Replacement####'
==> azure-arm:  -> DeploymentName    : 'pkrdp5u8k15bqf0'
==> azure-arm: Deploying deployment template ...
==> azure-arm:  -> ResourceGroupName : '###Resource Group Replacement####'
==> azure-arm:  -> DeploymentName    : 'kvpkrdp5u8k15bqf0'
==> azure-arm: Getting the certificate's URL ...
==> azure-arm:  -> Key Vault Name        : 'pkrkv5u8k15bqf0'
==> azure-arm:  -> Key Vault Secret Name : 'packerKeyVaultSecret'
==> azure-arm:  -> Certificate URL       : 'https://pkrkv5u8k15bqf0.vault.azure.net/secrets/packerKeyVaultSecret/d1a2c8fe28c34b73b65bb53b7a9ea34f'
==> azure-arm: Setting the certificate's URL ...
==> azure-arm: Validating deployment template ...
==> azure-arm:  -> ResourceGroupName : '###Resource Group Replacement####'
==> azure-arm:  -> DeploymentName    : 'pkrdp5u8k15bqf0'
==> azure-arm: Deploying deployment template ...
==> azure-arm:  -> ResourceGroupName : '###Resource Group Replacement####'
==> azure-arm:  -> DeploymentName    : 'pkrdp5u8k15bqf0'
==> azure-arm: Getting the VM's IP address ...
==> azure-arm:  -> ResourceGroupName   : '###Resource Group Replacement####'
==> azure-arm:  -> PublicIPAddressName : 'pkrip5u8k15bqf0'
==> azure-arm:  -> NicName             : 'pkrni5u8k15bqf0'
==> azure-arm:  -> Network Connection  : 'PrivateEndpoint'
==> azure-arm:  -> IP Address          : '10.236.8.92'
==> azure-arm: Waiting for WinRM to become available...
    azure-arm: #< CLIXML
    azure-arm: WinRM connected.
    azure-arm: <Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"><Obj S="progress" RefId="0"><TN RefId="0"><T>System.Management.Automation.PSCustomObject</T><T>System.Object</T></TN><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj><Obj S="progress" RefId="1"><TNRef RefId="0" /><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj></Objs>
==> azure-arm: Connected to WinRM!
==> azure-arm: Provisioning with Powershell...

您可以很清楚地看到它已通过winrm成功连接到新建的VM。以下是我用于打包程序的脚本：变量

{
  "tenant_id": "###Tenant ID Replacement ###",
  "subscription_id": "###Subscription ID Replacement",
  "region_name": "canadacentral",
  "virtual_network_rg": "###VNET ResourceGroupName replacement ####",
  "virtual_network_name": "###VNET Name Replacement ####",
  "virtual_network_subnet_name": "Presentation",
  "build_resource_group_name": "###Resource Group Replacement####",
  "image_resource_group_Name": "###Resource Group Replacement####",
  "chef_server": "https://chefserver01....",
  "object_id": "f3b76eb8-87e6-42d7-9d12-dafa5b124d90",
  "chef_org_url": "https://artifactory....",
  "chef_org_name": "org name replacement",
  "chef_client_binary": "https://artifactory.......",
  "chef_first_boot": "https://artifactory.......",
  "chef_client_config": "https://artifactory.......",
  "wsus_server": "http://WSUS....",
  "wsus_script": "https://artifactory......",
  "wsus_task": "https://artifactory.....",
  "chef_client_bootstrap": "https://artifactory.....",
  "chef_checkin_task": "https://artifactory......",
  "chef_checkin_xml": "https://artifactory......",
  "image_build_number": "1811261812",
  "client_id": "###Client ID replacement",
  "client_secret": "###Client Secret replacement"
}

windows.json

{
  "variables": {
    "client_secret": "{{user `client_secret`}}"
  },  
  "builders": [{
  "type": "azure-arm",
  "client_id": "{{user `client_id`}}",
  "client_secret": "{{user `client_secret`}}",
  "tenant_id": "{{user `tenant_id`}}",
  "subscription_id": "{{user `subscription_id`}}",
  "object_id": "{{user `object_id`}}",
  "build_resource_group_name": "{{user `build_resource_group_name`}}",

  "managed_image_resource_group_name": "{{user `image_resource_group_Name`}}",
  "managed_image_name": "Windows2016-dev-{{user `image_build_number`}}",

  "os_type": "Windows",
  "image_publisher": "MicrosoftWindowsServer",
  "image_offer": "WindowsServer",
  "image_sku": "2016-Datacenter",
  "image_version":"latest",
  "os_disk_size_gb":"128", 
  "vm_size": "Standard_E4S_v3",
  "virtual_network_resource_group_name":"{{user `virtual_network_rg`}}",
  "virtual_network_name":"{{user `virtual_network_name`}}",
  "virtual_network_subnet_name":"{{user `virtual_network_subnet_name`}}",



  "communicator": "winrm",
  "winrm_use_ssl": "true",
  "winrm_insecure": "true",
  "winrm_timeout": "3m",
  "winrm_username": "packer",

  "azure_tags": {
      "dept": "Platform Engineering",
      "task": "Daily Image Build",
      "CostCenter": "3357",
      "UseCase": "Standard Image Builds - GSPE",
      "ReleaseDate": "{{isotime `2006-01-02`}}"
  }
  }],
  "provisioners": [ 
    {
    "type": "powershell",
    "environment_vars": [
      "chef_org_url={{user `chef_org_url`}}",
      "chef_org_name={{user `chef_org_name`}}",
      "chef_client_binary={{user `chef_client_binary`}}",
      "chef_client_config={{user `chef_client_config`}}",
      "chef_first_boot={{user `chef_first_boot`}}",
      "chef_client_bootstrap={{user `chef_client_bootstrap`}}",
      "wsus_server={{user `wsus_server`}}",
      "wsus_script={{user `wsus_script`}}",
      "wsus_task={{user `wsus_task`}}",
      "chef_setrunonce={{user `chef_setrunonce`}}",
      "chef_server={{user `chef_server`}}",
      "chef_checkin_task={{user `chef_checkin_task`}}",
      "chef_checkin_xml={{user `chef_checkin_xml`}}"

    ],  
    "inline": [
      "$chef_org_client=$ENV:chef_org_name + '-validator'",
      "$chef_org_validator_filename=$chef_org_client + '.pem'",
      "$chef_org_validator_local='c:\\chef\\' + $chef_org_validator_filename",
      "$chef_org_validator_url=$ENV:chef_org_url + $chef_org_validator_filename",


      "Write-Host Setting TLS 1.2",
      "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12",

      "Write-Host Download Chef Client, Org Validator and First Boot",
      "Write-Host Full ORG Validator PEM FILE - $chef_org_validator_url",
      "Write-Host Chef Org Name - $env:chef_org_name",
      "Write-Host WSUS Server Name - $env:wsus_server",
      "Write-Host Chef Server - $env:chef_server",
      "md $env:windir\\Temp\\packer-chef-client -Force",
      "md $env:SystemDrive\\chef\\ -Force",
      "iwr -Uri $chef_org_validator_url -Outfile $chef_org_validator_local",
      "iwr -Uri $ENV:chef_client_binary -Outfile $env:windir\\Temp\\packer-chef-client\\chef.msi",
      "iwr -Uri $ENV:chef_client_config -Outfile $env:SystemDrive\\chef\\client.rb",
      "iwr -Uri $ENV:chef_first_boot -Outfile $env:SystemDrive\\chef\\first-boot.json",
      "iwr -Uri $ENV:chef_client_bootstrap -Outfile $env:windir\\Temp\\packer-chef-client\\chef_client_bootstrap.ps1",
      "iwr -Uri $ENV:chef_checkin_task -Outfile $env:windir\\Temp\\packer-chef-client\\chef_checkin_task.ps1",
      "iwr -Uri $ENV:chef_checkin_xml -Outfile $env:windir\\Temp\\packer-chef-client\\ChefRunOnce.xml",
      "iwr -Uri $ENV:wsus_script -Outfile $env:windir\\Temp\\packer-chef-client\\WSUSUpdate.ps1",
      "iwr -Uri $ENV:wsus_task -Outfile $env:windir\\Temp\\packer-chef-client\\DownloadStageChef-modded.ps1",

      "Write-Host Install Chef Client",
      "Start-Process 'msiexec' -ArgumentList '/qb /i C:\\Windows\\Temp\\packer-chef-client\\chef.msi' -NoNewWindow -Wait",

      "Write-Host Set Regional WSUS Server",
      "reg add 'HKEY_LOCAL_MACHINE\\SOFTWARE\\Policies\\Microsoft\\Windows\\WindowsUpdate' /v WUServer /t REG_SZ /d $ENV:wsus_server /f",
      "reg add 'HKEY_LOCAL_MACHINE\\SOFTWARE\\Policies\\Microsoft\\Windows\\WindowsUpdate' /v WUStatusServer /t REG_SZ /d $ENV:wsus_server /f",

      "Write-Host Run WSUS Update",
      "C:\\Windows\\Temp\\packer-chef-client\\DownloadStageChef-modded.ps1",

      "Write-Host Chef RunOnce Task",
      "C:\\Windows\\Temp\\packer-chef-client\\chef_checkin_task.ps1 -chef_org_name $env:chef_org_name -chef_server $env:chef_server"
    ]
  },
  {
    "type": "powershell",
    "inline": [
      "if( Test-Path $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml ){ rm $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml -Force}",
      "& $env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /quiet /quit",
      "while($true) { $imageState = Get-ItemProperty HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\State | Select ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE') { Write-Output $imageState.ImageState; Start-Sleep -s 10  } else { break } }"
    ]
  }
]
}

Answer 3

我后来发现，winrm我们需要添加一个证书才能启用winrm。并且要添加证书，我们需要首先创建一个密钥库和存储帐户。 Packer为我们完成了这些步骤，而terraform则没有。因此，我们必须在terraform模板中构建这些步骤以启用证书，然后启用winrm。倡议是引导Windows vm，而不必创建诸如存储帐户或密钥存储之类的其他资源，因为我需要在配置后清理它们，这会使模板复杂化。如果仍然必须创建其他资源，则我更喜欢使用Azure VM扩展，因为它不需要在NSG上启用winrm端口5985，这是另一个重要的管理问题。

Terraform设置程序无法对Azure上的新建Windows VM竞争

3 个答案: