如何处理配置单元中的结构数组

时间:2017-03-23 06:10:56

标签: hadoop hive hiveql apache-hive

我有一个下面给出的json文件,我想通过使用jsonserde创建一个外部表来使用hive访问它。

  

{" ResponseCode":" 1","响应":"找到数据","数据" :[{"季节":" RABI""扇区#34;:"园艺""分类":&# 34; Fruit"," Crop":" Mango"," QueryType":"文化实践"," QueryText&# 34;:"如何控制芒果中的花落?",#34; KCCAns":"建议喷洒到planofix 5-7 mili / pump",&# 34; StateName":" ANDHRA PRADESH"," DistrictName":" NELLORE"," BlockName":" BALAYAPALLE& #34;," SubmitDate":" 11/1/2016 9:05:27 AM"},{" Season":" RABI&#34 ;,"扇区#34;:"农业与#34;"分类":"其他""作物":&#34 ;其他"," QueryType":"杂草管理"," QueryText":" eucalyptus中的杂草管理"," KCCAns":"建议每次播放GLYPOSATE @ 2 LITERS",#34; StateName":" ANDHRA PRADESH"," DistrictName": "内洛尔""块名称":" MARRIPADU "," SubmitDate":" 11/1/2016 9:07:04 AM"},{" Season":" RABI&# 34;,"扇区#34;:"园艺""分类":"蔬菜""作物":&# 34; Bhindi(Okra / Ladysfinger)"," QueryType":" \ tPlant Protection \ t"," QueryText":" OKRA MITE损害"," KCCAns":"推荐喷涂DICOFOL 1 LITER / 200 LITERS OF WATER / ACRE"," StateName":" ANDHRA PRADESH& #34;,"区域名称":" NELLORE"," BlockName":" KOVUR"," SubmitDate": " 11/1/2016 3:11:59 PM"},{" Season":" RABI"," Sector":&# 34;农业","类别":"谷物","作物":"帕迪(Dhan)"," ; QueryType":"杂草管理"," QueryText":"稻谷杂草管理15-20天作物"," KCCAns" :"建议喷洒CYHALOPOP-P-BUTYL 250 ML PERRE"," StateName":" ANDHRA PRADESH"," DistrictName":" NELLORE"," BlockName&# 34;:" BALAYAPALLE"," SubmitDate":" 11/3/2016 2:11:17 PM"},{" Season&#34 ;:"夏播""扇区#34;:"农业与#34;"分类":"其他"&#34 ;裁剪":"其他"," QueryType":"天气"," QueryText":"天气报告&#34 ;," KCCAns":"天气报告联系18004253141,08912543031"," StateName":" ANDHRA PRADESH"," DistrictName&# 34;:" NELLORE"," BlockName":" KALIGIRI"," SubmitDate":" 11/3/2016 5 :22:22 PM"},{"赛季":" RABI"," Sector":"农业",&#34 ;类别":"豆类","作物":"木豆(红克/ arhar / tur)"," QueryType&#34 ;:"营养管理"," QueryText":& #34; RED GRAM NUTRIENT MANAGEMENT"," KCCAns":"建议每次喷洒13-045一公斤",#34; StateName":&#34 ; ANDHRA PRADESH"," DistrictName":" NELLORE"," BlockName":" ANUMASAMUDRAMPETA"," SubmitDate&# 34;:" 11/3/2016 7:25:10 PM"},{" Season":" RABI"," Sector" :"农业","类别":"豆类","作物":" Pigeon pea(红克/ arhar / tur )"," QueryType":"营养管理"," QueryText":" RED GRAM NUTRIENT MANAGEMENT"," KCCAns":"建议喷涂13-0-45 ONE KG / ACRE"," StateName":" ANDHRA PRADESH"," DistrictName&# 34;:" NELLORE"," BlockName":" ANUMASAMUDRAMPETA"," SubmitDate":" 11/3/2016 7 :30:02 PM"},{" Season":" RABI"," Sector":" HORTICULTURE",&#34 ;分类":"水果" ,"裁剪":" Citrus"," QueryType":" \ tPlant Protection \ t"," QueryText": " CITRUS SUCKING PEST"," KCCAns":"推荐用于DIMETHOATE 400 ML / ACRE 200 LITER WATER"," StateName":&#34 ; ANDHRA PRADESH"," DistrictName":" NELLORE"," BlockName":" KODAVALUR"," SubmitDate&# 34;:" 11/4/2016 8:48:03 AM"},{" Season":" RABI"," Sector" :"园艺""分类":"水果""作物":"柑橘"" QueryType":" \ tPlant Protection \ t"," QueryText":" CITRUS SUCKING PEST"," KCCAns":&#34 ;推荐用于DIMETHOATE 400 ML / ACRE 200 LITER WATER" StateName":" ANDHRA PRADESH"," DistrictName":" NELLORE&#34 ;," BlockName":" KODAVALUR"," SubmitDate":" 11/4/2016 8:48:04 AM"}, {"季&# 34;:"夏播""扇区#34;:"农业与#34;"分类":"其他"&# 34;裁剪":"其他"," QueryType":"天气"," QueryText":"天气预报&# 34;," KCCAns":"在今天下午有阵雨或雷雨的相当多云"," StateName":" ANDHRA PRADESH",& #34;区域名称":" NELLORE"," BlockName":" VINJAMUR"," SubmitDate":" 11 / 4/2016 3:14:03 PM"},{"赛季":" KHARIF"," Sector":"农业和#34 ;,"分类":"其他""作物":"其他""查询类型":&#34 ;天气"," QueryText":"天气转播"," KCCAns":"在今天下午有阵雨或雷雨的相当多云和#34 ;," StateName":" ANDHRA PRADESH"," DistrictName":" NELLORE"," BlockName":&# 34; MARRIPADU "," SubmitDate":" 11/4/2016 5:18:50 PM"},{" Season":" KHARIF&# 34;,"扇区#34;:"农业与#34;"分类":"其他""作物":&# 34;其他"," QueryType":" \ tPlant Protection \ t"," QueryText":" BENGAL GRAM BEST VARIETY", " KCCAns":" BENGAL GRAM BEST VARIETY:JG-11,LAM SENAGA(LBEG-7),ICCV-10"," StateName":" ANDHRA PRADESH"," DistrictName":" NELLORE"," BlockName":" PELLAKUR"," SubmitDate&#34 ;:" 11/4/2016 7:57:44 PM"}]}

我创建了下表来访问上面的json数据。

create EXTERNAL table kccdata1 (
Response string,
data array<struct<
Season:STRING,
Sector:STRING,
Category:STRING,
Crop:STRING,
QueryType:STRING,
QueryText:string,
KCCAns:string,
StateName:string,
DistrictName:string,
BlockName:string,
SubmitDate:string>>) 
row format serde "org.apache.hive.hcatalog.data.JsonSerDe" 
location "/user/hduser/jsondata";

此表将每个结构作为每个记录的值。我可以使用下面的查询来访问它中的元素。

select data[0].querytype from kccdata1;

select data.season[1] from kccdata1;

但是,如果我想找到像crop =其他所有扇区的东西,我无法使用此表格来完成。 对此有任何建议都会有很大帮助。感谢。

1 个答案:

答案 0 :(得分:6)

横向视图外联线

select  k.response
       ,d.* 

from    kccdata1 k 
        lateral view outer inline (k.data) d
;
+------------+----------+--------------+------------+---------------------------------+---------------------+------------------------------------------+-----------------------------------------------------------------------------+----------------+----------------+--------------------+----------------------+
| k.response | d.season |   d.sector   | d.category |             d.crop              |     d.querytype     |               d.querytext                |                                  d.kccans                                   |  d.statename   | d.districtname |    d.blockname     |     d.submitdate     |
+------------+----------+--------------+------------+---------------------------------+---------------------+------------------------------------------+-----------------------------------------------------------------------------+----------------+----------------+--------------------+----------------------+
| Data Found | RABI     | HORTICULTURE | Fruits     | Mango                           | Cultural Practices  | how to control flower drop in mango ?    | recommended for spray to planofix 5-7 mili/pump                             | ANDHRA PRADESH | NELLORE        | BALAYAPALLE        | 11/1/2016 9:05:27 AM |
| Data Found | RABI     | AGRICULTURE  | Others     | Others                          | Weed Management     | weed management in eucalyptus            | RECOMMENDED TO SPRAY GLYPOSATE @ 2 LITRES PER ACRE                          | ANDHRA PRADESH | NELLORE        | MARRIPADU          | 11/1/2016 9:07:04 AM |
| Data Found | RABI     | HORTICULTURE | Vegetables | Bhindi(Okra/Ladysfinger)        | Plant Protection    | OKRA MITE DAMAGE                         | RECOMMENDED TO SPRAY DICOFOL 1 LITER /200 LITRES OF WATER /ACRE             | ANDHRA PRADESH | NELLORE        | KOVUR              | 11/1/2016 3:11:59 PM |
| Data Found | RABI     | AGRICULTURE  | Cereals    | Paddy (Dhan)                    | Weed Management     | paddy weed management at 15-20 days crop | RECOMMENDED TO SPRAY CYHALOPOP-P- BUTYL 250 ML PER ACRE                     | ANDHRA PRADESH | NELLORE        | BALAYAPALLE        | 11/3/2016 2:11:17 PM |
| Data Found | KHARIF   | AGRICULTURE  | Others     | Others                          | Weather             | WEATHER REPORT                           | WEATHER REPORT CONTACT 18004253141 ,08912543031                             | ANDHRA PRADESH | NELLORE        | KALIGIRI           | 11/3/2016 5:22:22 PM |
| Data Found | RABI     | AGRICULTURE  | Pulses     | Pigeon pea (red gram/arhar/tur) | Nutrient Management | RED GRAM NUTRIENT MANAGEMENT             | RECOMMENDED TO SPRAY 13-045 ONE KG PER ACRE                                 | ANDHRA PRADESH | NELLORE        | ANUMASAMUDRAMPETA  | 11/3/2016 7:25:10 PM |
| Data Found | RABI     | AGRICULTURE  | Pulses     | Pigeon pea (red gram/arhar/tur) | Nutrient Management | RED GRAM NUTRIENT MANAGEMENT             | RECOMMENDED TO SPRAY 13-0-45 ONE KG/ACRE                                    | ANDHRA PRADESH | NELLORE        | ANUMASAMUDRAMPETA  | 11/3/2016 7:30:02 PM |
| Data Found | RABI     | HORTICULTURE | Fruits     | Citrus                          | Plant Protection    | CITRUS SUCKING PEST                      | RECOMMENDED FOR DIMETHOATE 400 ML / ACRE 200 LITER WATER                    | ANDHRA PRADESH | NELLORE        | KODAVALUR          | 11/4/2016 8:48:03 AM |
| Data Found | RABI     | HORTICULTURE | Fruits     | Citrus                          | Plant Protection    | CITRUS SUCKING PEST                      | RECOMMENDED FOR DIMETHOATE 400 ML / ACRE 200 LITER WATER                    | ANDHRA PRADESH | NELLORE        | KODAVALUR          | 11/4/2016 8:48:04 AM |
| Data Found | KHARIF   | AGRICULTURE  | Others     | Others                          | Weather             | weather report                           | Considerable cloudiness with a shower or thunderstorm around this afternoon | ANDHRA PRADESH | NELLORE        | VINJAMUR           | 11/4/2016 3:14:03 PM |
| Data Found | KHARIF   | AGRICULTURE  | Others     | Others                          | Weather             | WEATHER RTEPORT                          | Considerable cloudiness with a shower or thunderstorm around this afternoon | ANDHRA PRADESH | NELLORE        | MARRIPADU          | 11/4/2016 5:18:50 PM |
| Data Found | KHARIF   | AGRICULTURE  | Others     | Others                          | Plant Protection    | BENGAL GRAM BEST VARIETY                 | BENGAL GRAM BEST VARIETY : J.G-11, LAM SENAGA (LBEG-7),ICCV-10              | ANDHRA PRADESH | NELLORE        | PELLAKUR           | 11/4/2016 7:57:44 PM |
+------------+----------+--------------+------------+---------------------------------+---------------------+------------------------------------------+-----------------------------------------------------------------------------+----------------+----------------+--------------------+----------------------+