Question

在Azure数据工厂（V1）中，我能够创建幻灯片并将输出存储到特定文件夹（即{Year} / {Month} / {Day}。请参阅下面的代码。

如何在Azure Data Factory V2中创建相同类型的切片？我确实发现你必须创建一个参数。是的，我无法弄清楚如何传递参数。

 "folderPath": "@{dataset().path}",
"parameters": {
  "path": {
    "type": "String"

这是原始的ADF V1代码。

{
    "name": "EMS_EMSActivations_L1_Snapshot",
    "properties": {
        "published": false,
        "type": "AzureDataLakeStore",
        "linkedServiceName": "SalesIntelligence_ADLS_LS",
        "typeProperties": {
            "fileName": "EMS.FACT_EMSActivations_WTA.tsv",
            "folderPath": "/Snapshots/EMS/FACT_EMSActivations_WTA/{Year}/{Month}/{Day}",
            "format": {
                "type": "TextFormat",
                "rowDelimiter": "␀",
                "columnDelimiter": "\t",
                "nullValue": "#NULL#",
                "quoteChar": "\""
            },
            "partitionedBy": [
                {
                    "name": "Year",
                    "value": {
                        "type": "DateTime",
                        "date": "SliceStart",
                        "format": "yyyy"
                    }
                },
                {
                    "name": "Month",
                    "value": {
                        "type": "DateTime",
                        "date": "SliceStart",
                        "format": "MM"
                    }
                },
                {
                    "name": "Day",
                    "value": {
                        "type": "DateTime",
                        "date": "SliceStart",
                        "format": "dd"
                    }
                },
                {
                    "name": "Hour",
                    "value": {
                        "type": "DateTime",
                        "date": "SliceStart",
                        "format": "HH"
                    }
                },
                {
                    "name": "Minute",
                    "value": {
                        "type": "DateTime",
                        "date": "SliceStart",
                        "format": "mm"
                    }
                }
            ]
        },
        "availability": {
            "frequency": "Day",
            "interval": 1
        }
    }
}

Answer 1

以下是将数据从SQL导入ADL时创建动态文件夹路径的方法。查看folderPath行。

{
    "name": "EBC_BriefingActivitySummary_L1_Snapshot",
    "properties": {
        "linkedServiceName": {
            "referenceName": "SIAzureDataLakeStore",
            "type": "LinkedServiceReference"
        },
        "type": "AzureDataLakeStoreFile",
        "typeProperties": {
            "format": {
                "type": "TextFormat",
                "columnDelimiter": ",",
                "rowDelimiter": "",
                "nullValue": "\\N",
                "treatEmptyAsNull": false,
                "firstRowAsHeader": false
            },
            "fileName": {
                "value": "EBC.rpt_BriefingActivitySummary.tsv",
                "type": "Expression"
            },
            "folderPath": {
                "value": "@concat('/Snapshots/EBC/rpt_BriefingActivitySummary/', formatDateTime(pipeline().parameters.scheduledRunTime, 'yyyy'), '/', formatDateTime(pipeline().parameters.scheduledRunTime, 'MM'), '/', formatDateTime(pipeline().parameters.scheduledRunTime, 'dd'), '/')",
                "type": "Expression"
            }
        }
    }
}

Answer 2

第1步： 在folderpath中使用WindowStartTime / WindowEndTime

"folderPath": {
"value": "<<path>>/@{formatDateTime(pipeline().parameters.windowStart,'yyyy')}-@{formatDateTime(pipeline().parameters.windowStart,'MM')}-@{formatDateTime(pipeline().parameters.windowStart,'dd')}/@{formatDateTime(pipeline().parameters.windowStart,'HH')}/",
"type": "Expression"
}

第2步：添加管道JSON

"parameters": {
    "windowStart": {
        "type": "String"
    },
    "windowEnd": {
        "type": "String"
    }
}

步骤3：在TumblingWindow触发器中添加运行参数（这在步骤2中提到）

   "parameters": {
        "windowStart": {
            "type": "Expression",
            "value": "@trigger().outputs.windowStartTime"
        },
        "windowEnd": {
            "type": "Expression",
            "value": "@trigger().outputs.windowEndTime"
        }
    }

有关详细信息，请参阅

请参阅此链接。

https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/data-factory/how-to-create-tumbling-window-trigger.md

Azure Data Factory V2数据集动态文件夹

2 个答案: