蜂巢collect_set

Question

我正在处理大型数据集上的蜂巢，我有带有列数组的表，列的内容如下。

["20190302Prod4"
"20190303Prod1"
"20190303Prod4"
"20190304Prod4"
"20190305Prod3"
"20190307Prod4"
"20190308Prod4"
"20190309Prod4"
"20190310Prod2"
"20190311Prod1"
"20190311Prod4"
"20190312Prod1"
"20190312Prod4"
"20190313Prod2"
"20190313Prod1"
"20190313Prod4"
"20190314Prod4"
"20190315Prod4"
"20190316Prod4"
"20190317Prod1"
"20190317Prod4"]

我需要按照产品的升序日期进行设置，例如我需要从array修剪日期并应用collect_set才能获得低于结果的结果。

["Prod4",
"Prod1",
"Prod3",
"Prod2"]

Answer 1

展开数组，删除日期（字符串开头的数字），使用collect_set进行汇总：

with mydata as (--use your table instead of this
select array(
"20190302Prod4",
"20190303Prod1",
"20190303Prod4",
"20190304Prod4",
"20190305Prod3",
"20190307Prod4",
"20190308Prod4",
"20190309Prod4",
"20190310Prod2",
"20190311Prod1",
"20190311Prod4",
"20190312Prod1",
"20190312Prod4",
"20190313Prod2",
"20190313Prod1",
"20190313Prod4",
"20190314Prod4",
"20190315Prod4",
"20190316Prod4",
"20190317Prod1",
"20190317Prod4"
) myarray
)

select collect_set(regexp_extract(elem,'^\\d*(.*?)$',1)) col_name 
  from mydata a --Use your table instead  
       lateral view outer explode(myarray) s as elem;

结果：

col_name    
["Prod4","Prod1","Prod3","Prod2"]   

另一种可能的方法是首先连接数组，从字符串中删除日期，分割以获取数组。不幸的是，我们仍然需要爆炸以执行collect_set来删除重复项（例如，使用与WITH MYDATA CTE相同的示例）：

select collect_set(elem) col_name        
  from mydata a --Use your table instead 
       lateral view outer explode(split(regexp_replace(concat_ws(',',myarray),'(^|,)\\d{8}','$1'),',')) s as elem
;