如何确定执行配置单元查询所需的作业总数

时间:2019-02-08 09:12:56

标签: hadoop hive hiveql

有没有一种方法可以识别执行查询所需的作业总数。

例如,在下面的2个查询中,联接和子查询的数量相同,但是一个查询将需要2个工作,而其他查询则需要3个工作

 select t1.item_dim_key hive, t2.item_dim_key as monet 
   from ext_dist_it_dim_key t1 
        left outer join (select distinct item_dim_key from PO_ITEM_DIM) t2 on t1.item_dim_key=t2.item_dim_key 
   where t2.item_dim_key is null;

WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Query ID = 20190208020329_258ee4c0-5819-4842-b479-d549c82a0779
**Total jobs = 3**

hive> select t1.item_dim_key hive, t2.item_dim_key as monet 
       from (select distinct item_dim_key from PO_ITEM_DIM) t1 
            left outer join ext_dist_it_dim_key t2 on t1.item_dim_key=t2.item_dim_key 
      where t2.item_dim_key is null;

WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Query ID = 20190208020624_9ea3dc20-ffc8-4461-9516-7a4770d1dd6b
**Total jobs = 2**

是否可以知道执行一个查询需要多少工作? 计算作业数量需要哪些参数。

谢谢

1 个答案:

答案 0 :(得分:1)

使用EXPLAIN,它显示查询执行计划。只有计划才能肯定地帮助回答这个问题。根据统计信息或表(文件)的大小,Optimizer可以将某些联接转换为映射联接等。