在Hive中为什么我不允许在动态分区下嵌套静态分区?
例如允许以下
INSERT OVERWRITE TABLE T PARTITION (ds='2010-03-03', hr)
SELECT key, value, /*ds,*/ hr FROM srcpart WHERE ds is not null and hr>10;
但不允许这样做
INSERT OVERWRITE TABLE T PARTITION (ds, hr = 11)
SELECT key, value, ds/*, hr*/ FROM srcpart WHERE ds is not null and hr=11;
我发现官方维基页面解释(如下所示)不足。更喜欢基础地图缩小级别的逻辑解释或解释。
SP is a subpartition of a DP: should throw an error because partition column order determins directory hierarchy. We cannot change the hierarchy in DML
答案 0 :(得分:0)
这是一个Hive设计问题(specified here):
如果有多个分区列,则其顺序为 因为它转换为HDFS中的目录结构:
forall v from G: if(min(d[n] + w(n, v) | forall n from incoming_neighbours(v)) != d[n]): //something is wrong
表示目录结构partitioned by (ds string, dept int)
。在涉及分区表的DML或DDL中, 如果指定了分区列的子集(静态),我们应该 如果动态分区列较低,则抛出错误。
示例:
ds=2009-02-26/dept=2