ADF映射数据流-接收器转换动态分区数

Question

我有以下表达式可将Sink转换中的“分区数”计算为动态内容，

toInteger (round( iif(toDecimal('5671478512', 38, 2) <= 104857600, toDecimal(1.00) , toDecimal('5671478512', 38, 2)/104857600) ) )

该表达式的结果必须为整数54，但由于某种原因，在ADF门户中进行调试时会引发错误。

当我在派生的列转换中尝试确切的表达式时，得到的期望值为54。

有什么想法为什么它在“分区数”中失败？但是在派生列中测试时可以使用

以下是我在“分区数”动态内容中添加表达式时得到的错误

collectPreviewData failure on job=e97f7e77-abae-41f2-95dd-7d2d0e03800b, jobState=Failed com.microsoft.dataflow.Issues: DF-SYS-01 - requirement failed: Number of partitions (0) must be positive. - Nonejava.lang.IllegalArgumentException: requirement failed: Number of partitions (0) must be positive.
    at scala.Predef$.require(Predef.scala:224)
    at org.apache.spark.sql.catalyst.plans.logical.RepartitionByExpression.<init>(basicLogicalOperators.scala:1123)
    at com.microsoft.dataflow.TransformPlanner$$anonfun$physicalPartitionPlan$1.apply(Transformer.scala:299)
    at com.microsoft.dataflow.TransformPlanner$$anonfun$physicalPartitionPlan$1.apply(Transformer.scala:283)
    at scala.collection.immutable.Stream.map(Stream.scala:418)
    at com.microsoft.dataflow.TransformPlanner$class.physicalPartitionPlan(Transformer.scala:283)
    at com.microsoft.dataflow.transformers.ExternalCodeGenerator.physicalPartitionPlan(External.scala:126)
    at com.microsoft.dataflow.FlowRunner$$anonfun$16.apply(FlowRunner.scala:237)
    at com.microsoft.dataflow.FlowRunner$$anonfun$16.apply(FlowRunner.scala:216)
    at scala.collection.Iterator$$anon$11.next(Iterator.scala:410)
    at scala.collection.TraversableOnce$class.collectFirst(TraversableOnce.scala:145)
    at scala.collection.SeqViewLike$AbstractTransformed.collectFirst(SeqViewLike.scala:37)
    at com.microsoft.dataflow.FlowRunner$.com$microsoft$dataflow$FlowRunner$$runner(FlowRunner.scala:309)
    at com.microsoft.dataflow.FlowRunner$$anonfun$runner$2.apply(FlowRunner.scala:178)
    at com.microsoft.dataflow.FlowRunner$$anonfun$runner$2.apply(FlowRunner.scala:173)
    at scala.util.Success.flatMap(Try.scala:231)
    at com.microsoft.dataflow.FlowRunner$.runner(FlowRunner.scala:173)
    at com.microsoft.dataflow.DataflowExecutor$$anonfun$6$$anonfun$apply$3$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$9$$anonfun$apply$10$$anonfun$apply$11$$anonfun$7.apply(DataflowExecutor.scala:119)
    at com.microsoft.dataflow.DataflowExecutor$$anonfun$6$$anonfun$apply$3$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$9$$anonfun$apply$10$$anonfun$apply$11$$anonfun$7.apply(DataflowExecutor.scala:106)
    at com.microsoft.dataflow.DataflowJobFuture$$anonfun$flowCode$1.apply(DataflowJobFuture.scala:66)
    at com.microsoft.dataflow.DataflowJobFuture$$anonfun$flowCode$1.apply(DataflowJobFuture.scala:66)
    at scala.Option.map(Option.scala:146)
    at com.microsoft.dataflow.DataflowJobFuture.flowCode$lzycompute(DataflowJobFuture.scala:66)
    at com.microsoft.dataflow.DataflowJobFuture.flowCode(DataflowJobFuture.scala:66)
    at com.microsoft.dataflow.DataflowJobFuture$$anonfun$start$1.apply$mcV$sp(DataflowJobFuture.scala:290)
    at com.microsoft.dataflow.DataflowJobFuture$$anonfun$start$1.apply(DataflowJobFuture.scala:287)
    at com.microsoft.dataflow.DataflowJobFuture$$anonfun$start$1.apply(DataflowJobFuture.scala:287)
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    at com.microsoft.dataflow.DataflowJobFuture$$anonfun$start$1.apply$mcV$sp(DataflowJobFuture.scala:315)
    at com.microsoft.dataflow.DataflowJobFuture$$anonfun$start$1.apply(DataflowJobFuture.scala:287)
    at com.microsoft.dataflow.DataflowJobFuture$$anonfun$start$1.apply(DataflowJobFuture.scala:287)
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Answer 1

根据@Mark Kromer的评论分享答案。 不幸的是，“我们不在评估Number of Partitions属性中的动态内容。”这被确认为Bug，产品团队正在积极致力于此修复。