Question

我正在将文件部署到蔚蓝的Datalake，但想将datetime stamp添加到filename。我创建了一个数据集，并创建了一个值为utcnow（）的参数。但是文件未部署并附加了时间戳。

{

"name": "tst",

"properties": {

    "linkedServiceName": {

        "referenceName": "LS_ADLS_AzureDataLakeStoreLinkedService",

        "type": "LinkedServiceReference"

    },

    "parameters": {

        "customVariable": {

            "type": "String",

            "defaultValue": "utcnow()"

        }

    },

    "type": "AzureDataLakeStoreFile",

    "typeProperties": {

        "format": {

            "type": "TextFormat",

            "columnDelimiter": "|",

            "rowDelimiter": "",

            "nullValue": "",

            "treatEmptyAsNull": true,

            "skipLineCount": 0,

            "firstRowAsHeader": true

        },

        "fileName": "abc-{customVariable}",

        "folderPath": "clusters/diageo-eun-analytics-nonprod-hdi-hd-nampolaris-dev01/hive/warehouse/Spirit_Deal/tst",

        "partitionedBy": [

            {

                "name": "customVariable",

                "value": {

                    "type": "DateTime",

                    "date": "SliceStart",

                    "format": "yyyyMMddHH"

                }

            }

        ]

    }

},

"type": "Microsoft.DataFactory/factories/datasets"

}

但是文件名现在附带时间戳，即abc- {customVariable}。请说明如何在ADV v2中添加

Answer 1

您绝对可以参数化文件名和路径。点您尝试@CONCAT。以下示例有效，展示了如何动态构建文件夹路径和文件名：

{
    "name": "TgtADLSSQL",
    "properties": {
        "linkedServiceName": {
            "referenceName": "Destination-DataLakeStore-gqf",
            "type": "LinkedServiceReference"
        },
        "folder": {
            "name": "Target"
        },
        "type": "AzureDataLakeStoreFile",
        "typeProperties": {
            "format": {
                "type": "ParquetFormat"
            },
            "fileName": {
                "value": "@CONCAT('Ingest_Date=', formatDateTime(trigger().startTime, 'yyyy-MM-dd/'),'Incremental-', pipeline().RunId, '.snappy.parquet')",
                "type": "Expression"
            },
            "folderPath": {
                "value": "/data/raw/corporate/sql/@{item().TABLE_NAME}",
                "type": "Expression"
             }
        }
    },
    "type": "Microsoft.DataFactory/factories/datasets"
}

另请参阅Dynamic folders in ADF V2。

Azure ADF v2-如何向要部署到Azure Data Lake的文件名添加/附加时间戳

1 个答案: