spark-sql -e执行不能有太多的count子句

时间:2016-12-21 07:55:17

标签: apache-spark-sql

我使用spark-sql -e在sql下面执行。 Spark返回了一个例外:

ERROR CodeGenerator: failed to compile: 
org.codehaus.janino.JaninoRuntimeException: 
agg_doAggregateWithKeys(): Operand stack inconsistent at offset 778: 
Previous size 1, now 0/* 001 */ public Object generate(Object[] references)

遵循GeneratedIterator的源代码

但是当我从thrifserver客户端或hive -e命令执行相同的查询时,一切都很好。

任何根本原因的想法??

select 
      to_date(action_time) as value_date,
      platform,
      count(distinct( case when action_name='home' then device_num else null end)) as ea_home_uv,
      count(distinct( case when action_name='search' then device_num else null end)) as ea_search_uv,
      count(distinct( case when action_name='searchResult' then device_num else null end)) as ea_searchResult_uv,
      count(distinct( case when action_name='addToCart' and pre_name='searchResult' and branch='search' then device_num else null end)) as ea_searchResult_addToCart_uv,
      count(distinct( case when action_name='detail' and pre_name='searchResult' then device_num else null end)) as ea_search_detail_uv,
      count(distinct( case when action_name='addToCart' and pre_name='detail' and branch='search' then device_num else null end)) as ea_search_detail_addToCart_uv,
      count(distinct( case when action_name='topCategory' then device_num else null end)) as ea_topCategory_uv,
      count(distinct( case when action_name='secCategory' then device_num else null end)) as ea_secCategory_uv,
      count(distinct( case when action_name='addToCart' and pre_name='secCategory' and branch='category' then device_num else null end)) as ea_secCategory_addToCart_uv,
      count(distinct( case when action_name='detail' and pre_name='secCategory' then device_num else null end)) as ea_category_detail_uv,
      count(distinct( case when action_name='addToCart' and pre_name='detail' and branch='category' then device_num else null end)) as ea_category_detail_addToCart_uv,
      count(distinct( case when action_name='H5Page' then device_num else null end)) as ea_H5Page_uv,
      count(distinct( case when action_name='addToCart' and pre_name='H5Page' then device_num else null end)) as ea_H5Page_addToCart_uv,
      count(distinct( case when action_name='detail' and pre_name='H5Page' then device_num else null end)) as ea_H5Page_detail_uv,
      count(distinct( case when action_name='addToCart' and pre_name='detail' and branch='h5' then device_num else null end)) as ea_H5Page_detail_addToCart_uv,
      count(distinct( case when action_name='detail' and pre_name='home' then device_num else null end)) as ea_detail_uv,
      count(distinct( case when action_name='addToCart' and pre_name='detail' and branch='home' then device_num else null end)) as ea_detail_addToCart_uv,
      count(distinct( case when action_name='salesList' then device_num else null end)) as ea_salesList_uv,
      count(distinct( case when action_name='detail' and pre_name='salesList' then device_num else null end)) as ea_salesList_detail_uv,
      count(distinct( case when action_name='addToCart' and pre_name='detail' and branch='sales' then device_num else null end)) as ea_salesList_detail_addToCart_uv,
      count(distinct( case when action_name='addToCart' and pre_name='salesList' then device_num else null end)) as ea_salesList_addToCart_uv,
      count(distinct( case when action_name='shoppingCart' then device_num else null end)) as ea_shoppingCart_uv,
      count(distinct( case when action_name='orderConfirming' then device_num else null end)) as ea_orderConfirming_uv,
      count(distinct( case when action_name='submitOrder' then device_num else null end)) as ea_submitOrder_uv,
      count(distinct( case when action_name='cashier' then device_num else null end)) as ea_cashier_uv,
      count(distinct( case when action_name='payment' then device_num else null end)) as ea_payment_uv,
      count(distinct( case when action_name='paid' then device_num else null end)) as ea_paid_uv,
      count( case when action_name='home' then 1 else null end) as ea_home_pv,
      count( case when action_name='search' then 1 else null end) as ea_search_pv,
      count( case when action_name='searchResult' then 1 else null end) as ea_searchResult_pv,
      count( case when action_name='addToCart' and pre_name='searchResult' and branch='search' then 1 else null end) as ea_searchResult_addToCart_pv,
      count( case when action_name='detail' and pre_name='searchResult' then 1 else null end) as ea_search_detail_pv,
      count( case when action_name='addToCart' and pre_name='detail' and branch='search' then 1 else null end) as ea_search_detail_addToCart_pv,
      count( case when action_name='topCategory' then 1 else null end) as ea_topCategory_pv,
      count( case when action_name='secCategory' then 1 else null end) as ea_secCategory_pv,
      count( case when action_name='addToCart' and pre_name='secCategory' and branch='category' then 1 else null end) as ea_secCategory_addToCart_pv,
      count( case when action_name='detail' and pre_name='secCategory' then 1 else null end) as ea_category_detail_pv,
      count( case when action_name='addToCart' and pre_name='detail' and branch='category' then 1 else null end) as ea_category_detail_addToCart_pv,
      count( case when action_name='H5Page' then 1 else null end) as ea_H5Page_pv,
      count( case when action_name='addToCart' and pre_name='H5Page' then 1 else null end) as ea_H5Page_addToCart_pv,
      count( case when action_name='detail' and pre_name='H5Page' then 1 else null end) as ea_H5Page_detail_pv,
      count( case when action_name='addToCart' and pre_name='detail' and branch='h5' then 1 else null end) as ea_H5Page_detail_addToCart_pv,
      count( case when action_name='detail' and pre_name='home' then 1 else null end) as ea_detail_pv,
      count( case when action_name='addToCart' and pre_name='detail' and branch='home' then 1 else null end) as ea_detail_addToCart_pv,
      count( case when action_name='salesList' then 1 else null end) as ea_salesList_pv,
      count( case when action_name='detail' and pre_name='salesList' then 1 else null end) as ea_salesList_detail_pv,
      count( case when action_name='addToCart' and pre_name='detail' and branch='sales' then 1 else null end) as ea_salesList_detail_addToCart_pv,
      count( case when action_name='addToCart' and pre_name='salesList' then 1 else null end) as ea_salesList_addToCart_pv,
      count( case when action_name='shoppingCart' then 1 else null end) as ea_shoppingCart_pv,
      count( case when action_name='orderConfirming' then 1 else null end ) as ea_orderConfirming_pv,
      count( case when action_name='submitOrder' then 1 else null end) as ea_submitOrder_pv,
      count( case when action_name='cashier' then 1 else null end) as ea_cashier_pv,
      count( case when action_name='payment' then 1 else null end) as ea_payment_pv,
      count( case when action_name='paid' then 1 else null end) as ea_paid_pv
  from 
      l2_ux_t_pv
  where 
      to_date(insert_time)>='2016-12-20'
  group by 
      to_date(action_time),
      platform

1 个答案:

答案 0 :(得分:0)

这是Janino编译器中的错误,已在Spark v2.1 +中修复,相应于:https://issues.apache.org/jira/browse/SPARK-15467

另请参阅: