Question

我有一个U-SQL脚本，它运行在存储在Data Lake中的许多JSON文件，当我针对单个文件运行脚本时，一切运行正常，但是打开它以对所有文件运行我每次都会收到以下错误，并且工作永远不会超过“准备”工作。阶段（花费大约30分钟）：

System: E_SYSTEM_COMPILER_COMPILERINTERNALERROR
SEVERITY: ERROR
DESCRIPTION: Compilation failed due to internal system error.
MESSAGE: Compilation failed due to internal system error. (255)
DETAILS: Compilation failed for Deep.; JobId: 6ddc1049-c064-46d4-    b31f-1073ffd2cc99; Duration: 00:31:10.4668250; Exit Code: 255; Configuration: kobo_vnext_signed_5395848; Runtime: kobo_vnext_signed_5395848; Output: Deleted file D:\data\CCS\jobs\6ddc1049-c064-46d4-b31f-1073ffd2cc99_v0\securityInfo.
Deleted file D:\data\CCS\jobs\6ddc1049-c064-46d4-b31f-1073ffd2cc99_v0\jobEncryptionKey.
Server Version: 10.2.2.0
Client Version: 10.2.2.0
Target: Cluster
Stat: FetchReal
SkipPath: False
Syntax Parsing finished.
Resource Resolution finished.
Input/Output Resolution finished.
HandleCodeBehind finished.
Wait until structured stream metadata loading end finished.
Binder Pass finished.
Semantic Analyzer finished.
Error Reporter finished.
OptTree Generator finished.

--==== CQO Optimization ====--
Traceflag 608 is ON
Traceflag 551 is ON
Traceflag 402 is ON
Traceflag 550 is ON
Traceflag 21 is ON
Traceflag 552 is ON
Traceflag 554 is ON
Traceflag 515 is ON
Traceflag 534 is ON
Traceflag 541 is ON
Traceflag 435 is ON
Traceflag 106 is ON
Traceflag 62 is ON
Traceflag 571 is ON
Traceflag 514 is ON
Traceflag 568 is ON
Traceflag 569 is ON
Traceflag 203 is ON
Traceflag 546 is ON
Traceflag 562 is ON
Traceflag 563 is ON
Traceflag 505 is ON
Traceflag 506 is ON
Traceflag 542 is ON
Polaris scale up query execution is not enabled
Timer[CQO - Init] = 0.0172413 s
Timer[CQO - CopyIn] = 0.154232 s
Timer[CQO - Search - Init] = 0.000229223 s
Timer[CQO - Search - NNF] = 0.0124772 s
Timer[CQO - Search - PreOptimization] = 0.000576905 s
Pruning partitions and streamsets
Simplifying relational tree

Timer[CQO - Total] = 749.761 s
CQO: Optimization failed.


;

RESOLUTION

在VS2017的诊断标签上，我也看到了：

Vertex ran too long: 00:31:10

是什么导致这个？我在努力解决这个问题

Answer 1

您可能想尝试FileSet选项（https://github.com/Azure/AzureDataLake/blob/master/docs/Release_Notes/2017/2017_04_24/USQL_Release_Notes_2017_04_24.md#input-file-set-scales-orders-of-magnitudes-better-opt-in-statement-is-now-provided）

几周前我遇到了类似的问题。我首先通过增加AU的数量来解决问题。但是，然后，使用

 SET @@FeaturePreviews = "FileSetV2Dot5:on";

有助于缩短编制时间，同时使用较少量的AU

在ADLA准备时顶点太长

1 个答案: