如何通过使用azuredatafactory将多个文件从azure blob加载到azure synapse(Dw)?

时间:2020-02-26 18:00:37

标签: azure-data-factory azure-container-service azure-synapse

我正在尝试使用azure数据工厂将多个文件从azure blob加载到azure sql dw。以下是我的代码。我正面临突出显示的错误。有人可以建议。 我在这里粘贴我的adf代码json。

我正在复制活动阶段低于“突出显示”。

{
    "name": "DelimitedText11",
    "properties": {
        "linkedServiceName": {
            "referenceName": "AzureBlobStorage2",
            "type": "LinkedServiceReference"
        },
        "parameters": {
            "FileName": {
                "type": "string"
            }
        },
        "annotations": [],
        "type": "DelimitedText",
        "typeProperties": {
            "location": {
                "type": "AzureBlobStorageLocation",
                "fileName": {
                    "value": "@dataset().FileName",
                    "type": "Expression"
                },
                "container": "input"
            },
            "columnDelimiter": ",",
            "escapeChar": "",
            "firstRowAsHeader": true,
            "quoteChar": ""
        },
        "schema": []
    },
    "type": "Microsoft.DataFactory/factories/datasets"
}

错误:

{
        "errorCode": "2200",
        "message": "ErrorCode=UserErrorMissingPropertyInPayload,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Required property 'fileName' is missing in payload.,Source=Microsoft.DataTransfer.ClientLibrary,'",
        "failureType": "UserError",
        "target": "Copy data1",
        "details": []
    }

1 个答案:

答案 0 :(得分:1)

如果要将多个文件从azure blob加载到Azure SQL数据仓库,请必须在源数据集中设置Wildcard file path。否则,您总是会收到错误2200:

{
        "errorCode": "2200",
        "message": "ErrorCode=UserErrorMissingPropertyInPayload,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Required property 'fileName' is missing in payload.,Source=Microsoft.DataTransfer.ClientLibrary,'",
        "failureType": "UserError",
        "target": "Copy data1",
        "details": []
    }

例如,我有两个具有相同架构的csv文件,并将它们加载到我的Azure SQL数据仓库表test中。

Csv文件:

enter image description here

源数据集:

enter image description here

源设置:选择源容器中的所有csv文件:

enter image description here

接收器数据集:

enter image description here

接收器设置:

enter image description here

映射: enter image description here

设置: enter image description here

执行管道并检查ADW中的数据: enter image description here

enter image description here

希望这会有所帮助。