Question

我有一个非常复杂的查询，我试图将其转换为与Hive一起使用。

具体地说，我将其作为一个Hive“步骤”在AWS EMR集群中运行。

我已经尝试过一些清理帖子的查询，只是保留了本质。

完整的错误消息是：

FAILED: SemanticException [Error 10128]: Line XX:XX Not yet supported place for UDAF 'COUNT'

行号指向select语句底部的COUNT：


    INSERT INTO db.new_table (
        new_column1,
        new_column2,
        new_column3,
        ... ,
        new_column20
    ) 
    SELECT MD5(COALESCE(TBL1.col1," ")||"_"||COALESCE(new_column5," ")||"_"||...) AS 
        new_col1,
        TBL1.col2,
        TBL1.col3,
        TBL1.col3 AS new_column3,
        TBL1.col4,
        CASE
            WHEN TBL1.col5 = …
            ELSE “some value”
        END AS new_column5,
        TBL1.col6,
        TBL1.col7,
        TBL1.col8,
        CASE
            WHEN TBL1.col9 = …
            ELSE "some value"
        END AS new_column9,
        CASE 
            WHEN TBL1.col10 = …
            ELSE "value"
        END AS new_column10,
        TBL1.col11,
        "value" AS new_column12,
        TBL2.col1,
        TBL2.col2,
        from_unixtime(…) AS new_column13,
        CAST(…) AS new_column14,
        CAST(…) AS new_column15,
        CAST(…) AS new_column16,
        COUNT(DISTINCT TBL1.col17) AS new_column17
    FROM db.table1 TBL1
    LEFT JOIN 
        db.table2 TBL2
            ON TBL1.col311 = TBL2.col311
    WHERE TBL1.col14 BETWEEN "low" AND "high"
        AND TBL1.col44 = "Y"
        AND TBL1.col55 = "N"
    GROUP BY 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20;

如果我遗漏了太多，请告诉我。

感谢您的帮助！

更新

事实证明，实际上我确实遗漏了太多信息。抱歉，那些已经尝试提供帮助的人...

我在上面做了更新。

按列删除第20组，例如：

GROUP BY 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19;

产生：Expression not in GROUP BY key '' ''

最新

按列删除第20个组，然后添加第一个，例如：

GROUP BY 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19;

生产：

Line XX:XX Invalid table alias or column reference 'new_column5':(possible column
 names are: TBL1.col1, TBL1.col2, (looks like all columns of TBL1), 
TBL2.col1, TBL2.col2, TBL2.col311)

第＃行使用SELECT语句引用该行。错误输出中仅列出了来自TBL2的那三列。

错误似乎指向COALESCE(new_column5)。请注意，我在CASE中运行的TBL 1 select中有一个AS new_column5语句。

Answer 1

您要在与计算列名称相同的子查询级别上寻址计算列名称new_column5。在Hive中这是不可能的。将其替换为计算本身，或使用上级子查询。

此：

 MD5(COALESCE(TBL1.col1," ")||"_"||COALESCE(CASE WHEN TBL1.col5 = … ELSE “some value” END," ")||"_"||...) AS new_col1,

代替此：

 MD5(COALESCE(TBL1.col1," ")||"_"||COALESCE(new_column5," ")||"_"||...) AS 
        new_col1,

AWS EMR Hive：UDAF'COUNT'尚不支持的位置

1 个答案: