Question

我要创建一个包含U-SQL脚本的管道，该管道将Azure DataLake存储中的多个日志文件合并为一个文件，我试图通过在存储中创建一个文本文件然后将其添加到管道中来做到这一点< strong> scriptpath ，但出现错误，我进行了搜索，发现管道不支持ADL，因此我想使用 script 在管道中编写U-SQL脚本属性。我尝试使用此脚本来执行此操作，但是出现错误，无法部署它，因此任何人都可以帮助您执行此操作？这是我的管道脚本：

    {
        "name": "RG-GatherData",
        "properties": {
            "description": "description",
            "activities": [
                {
                    "name": "DataLakeAnalyticsUSqlActivityTemplate",
                    "type": "DataLakeAnalyticsU-SQL",
                    "linkedServiceName": "AzureDataLakeAnalyticsLinkedService",

                    "typeProperties": {
                        "script": "
    @log = EXTRACT ["VersionID"] int,
               ["NodeName"] string,
               ["UpdateIng Area"] string,
               ["ActionDate"] string,
               ["UserName"] string,
               ["Code part Type"]  string,
               ["DocCode"] string,
               ["Header Entity Id"] string,
               ["Common Entity Id"] string,
               ["Attribute Name"] string,
               ["Latest Update Value"] string,
               ["Previous Update Value"] string  
          FROM @in
          USING Extractors.Csv(skipFirstNRows: 1);

OUTPUT @log
TO @out
USING Outputters.Csv();

                        ,
                        "degreeOfParallelism": 3,
                        "priority": 100,
                        "parameters": {
                            "in": "/RowLogs/InPut/RoyalGardens/{*}.csv",
                            "out": "/RowLogs/OutPut/RoyalGardens/Alllog.csv"
                        }
                    },
                    "policy": {
                        "concurrency": 1,
                        "executionPriorityOrder": "OldestFirst",
                        "retry": 3,
                        "timeout": "10:00:00"
                    },
                    "scheduler": {
                        "frequency": "Day",
                        "interval": 1
                    }
                }
            ],
            "start": "2018-09-20T00:06:00Z",
            "end": "2099-12-30T22:00:00Z"
        }
    }

Answer 1

将U-SQL脚本存储在Blob存储中，并通过Blob存储链接服务引用它。

在Data Factory Pipeline中运行DataLakeAnalyticsU-SQL脚本

1 个答案: