在Hive中转置数据

Question

我有一个Hive表，其中包含以下格式的数据

day       class  start_time  count  kpi1  kpi2  kpi3  kpi4  ...  kpi160
-----------------------------------------------------------------------
20161010  abc    00          12     1     0     null  0     ...

我想编写一个hive查询来获取以下格式的数据 使用某些计算，例如max，min和avg。

day       class  start_time  count  kpi_name  kpi_max   kpi_min   kpi_avg
-----------------------------------------------------------------------
20161010  abc    00          12     kpi1      max(kpi1) min(kpi1) avg(kpi1)
20161010  abc    00          12     kpi2      max(kpi2) min(kpi2) avg(kpi2)

请建议以此格式获取数据的解决方案。

感谢。

Answer 1

您需要将所有HTTP_USER_AGENT放入地图中，然后展开地图以创建一列，然后进行汇总。

示例：

数据：

kpi

<强>查询：

+---------+------+-----------+-------+-----+-----+------+------+------+------+
|day_     |class |start_time |count_ |kpi0 |kpi1 | kpi2 | kpi3 | kpi4 | kpi5 |
+---------+------+-----------+-------+-----+-----+------+------+------+------+
|20161010 |abc   |00         |12     |1    |2    |3     |8     |9     |6     |
+---------+------+-----------+-------+-----+-----+------+------+------+------+
|20161010 |abc   |00         |12     |4    |5    |null  |6     |10    |null  |
+---------+------+-----------+-------+-----+-----+------+------+------+------+

<强>输出：

SELECT day_
  , class
  , start_time
  , count_
  , kpi_type
  , MAX(vals) AS max_vals
  , MIN(vals) AS min_vals
  , AVG(vals) AS avg_vals
FROM (
  SELECT day_, class, start_time, count_, kpi_type, vals
  FROM database.table
  LATERAL VIEW EXPLODE(MAP('kpi0', kpi0
                         , 'kpi1', kpi1
                         , 'kpi2', kpi2
                         , 'kpi3', kpi3
                         , 'kpi4', kpi4
                         , 'kpi5', kpi5)) et AS kpi_type, vals ) x
GROUP BY day_, class_, start_time, count_, kpi_type

Answer 2

如果你想获得min，max，avg，你必须指定group By列，假设你想按天分组。

db.Phone.find({phone: /^\d{5}$/})