Question

我用ARM模板创建了一个Azure Data Lake gen 2。但是，现在我试图弄清楚如何在ARM中创建Data Lake File系统，但是似乎找不到实现此目的的API。这不可用，是否可以通过其他方式实现？

试图手动创建文件系统并在门户中导出模板，但是似乎看不到文件系统资源。只有这个警告。

因为这表明文件系统可能在API中以“ blobservices / containers”的形式进行监听，所以我试图将此资源添加到ARM模板中

 {
            "name": "[concat( parameters('DataLakeName'), '/default/input')]",
            "type": "Microsoft.Storage/storageAccounts/blobServices/containers",
            "dependsOn": [
                "[concat('Microsoft.Storage/storageAccounts/', parameters('DataLakeName'))]"
            ],
            "apiVersion": "2018-07-01",
            "properties": {
                "publicAccess": "None",
                "metadata": {}
            },
            "resources": []
        }

但是不幸的是，这无法正常工作，并提供了以下错误消息： Blob API is not yet supported for hierarchical namespace accounts. 这让我想到，甚至不可能通过ARM进行部署。有没有人已经尝试过这个？

我的datalake存储帐户的ARM模板资源块已完成作为上下文：

 {
            "name": "[parameters('DataLakeName')]",
            "type": "Microsoft.Storage/storageAccounts",
            "apiVersion": "2018-07-01",
            "location": "[parameters('location')]",
            "tags": {},
            "properties": {
                "accessTier": "[parameters('accessTier')]",
                "supportsHttpsTrafficOnly": "[parameters('supportsHttpsTrafficOnly')]",
                "isHnsEnabled": true
            },
            "resources": [
                {
                    "type": "providers/advancedThreatProtectionSettings",
                    "name": "Microsoft.Security/current",
                    "apiVersion": "2017-08-01-preview",
                    "dependsOn": [
                        "[resourceId('Microsoft.Storage/storageAccounts/', parameters('DataLakeName'))]"
                    ],
                    "properties": {
                        "isEnabled": true
                    }
                }
            ],
            "dependsOn": [],
            "sku": {
                "name": "[parameters('accountType')]"
            },
            "kind": "StorageV2"
        }

Answer 1

我在此github issue answer上找到了此解决方案。从一个发现还没有解决方案的家伙那里，所以这里要做的基本事情是手动调用rest api。说明和博客可以在此链接中找到： http://sql.pawlikowski.pro/2019/03/10/connecting-to-azure-data-lake-storage-gen2-from-powershell-using-rest-api-a-step-by-step-guide/

这是Powershell脚本，用于创建文件系统，以防万一该链接被弃用：

MichałPawlikowski的所有功劳，感谢您创建此脚本的过程很吸引人。

[CmdletBinding()]
Param(
  [Parameter(Mandatory=$true,Position=1)] [string] $StorageAccountName,
  [Parameter(Mandatory=$True,Position=2)] [string] $FilesystemName,
  [Parameter(Mandatory=$True,Position=3)] [string] $AccessKey
)

# Rest documentation:
# https://docs.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/filesystem/create

$date = [System.DateTime]::UtcNow.ToString("R") # ex: Sun, 10 Mar 2019 11:50:10 GMT

$n = "`n"
$method = "PUT"

$stringToSign = "$method$n" #VERB
$stringToSign += "$n" # Content-Encoding + "\n" +  
$stringToSign += "$n" # Content-Language + "\n" +  
$stringToSign += "$n" # Content-Length + "\n" +  
$stringToSign += "$n" # Content-MD5 + "\n" +  
$stringToSign += "$n" # Content-Type + "\n" +  
$stringToSign += "$n" # Date + "\n" +  
$stringToSign += "$n" # If-Modified-Since + "\n" +  
$stringToSign += "$n" # If-Match + "\n" +  
$stringToSign += "$n" # If-None-Match + "\n" +  
$stringToSign += "$n" # If-Unmodified-Since + "\n" +  
$stringToSign += "$n" # Range + "\n" + 
$stringToSign +=    
                    <# SECTION: CanonicalizedHeaders + "\n" #>
                    "x-ms-date:$date" + $n + 
                    "x-ms-version:2018-11-09" + $n # 
                    <# SECTION: CanonicalizedHeaders + "\n" #>

$stringToSign +=    
                    <# SECTION: CanonicalizedResource + "\n" #>
                    "/$StorageAccountName/$FilesystemName" + $n + 
                    "resource:filesystem"# 
                    <# SECTION: CanonicalizedResource + "\n" #>

$sharedKey = [System.Convert]::FromBase64String($AccessKey)
$hasher = New-Object System.Security.Cryptography.HMACSHA256
$hasher.Key = $sharedKey

$signedSignature = [System.Convert]::ToBase64String($hasher.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($stringToSign)))


$authHeader = "SharedKey ${StorageAccountName}:$signedSignature"

$headers = @{"x-ms-date"=$date} 
$headers.Add("x-ms-version","2018-11-09")
$headers.Add("Authorization",$authHeader)

$URI = "https://$StorageAccountName.dfs.core.windows.net/" + $FilesystemName + "?resource=filesystem"

Try {
    Invoke-RestMethod -method $method -Uri $URI -Headers $headers # returns empty response
}
catch {
    $ErrorMessage = $_.Exception.Message
    $StatusDescription = $_.Exception.Response.StatusDescription
    $false

    Throw $ErrorMessage + " " + $StatusDescription
}

Answer 2

签出az存储fs（https://docs.microsoft.com/en-us/cli/azure/storage/fs?view=azure-cli-latest）。这样，您就可以管理Azure Data Lake Gen2中的文件系统。

这样，您可以首先通过ARM启用具有分层名称空间的存储帐户，然后运行脚本来创建文件系统（脚本的执行也可以包含在ARM模板中-> https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/deployment-script-template?tabs=PowerShell < / p>

通过ARM模板在Azure数据湖存储帐户中创建文件系统

2 个答案: