Question

您好我有一个带有struct数组的avro架构，我可以将数据保存为avro。但是从

中检索数据时

array<struct<string, string>>

我无法排队。我单排获得的所有数据。

这是表格定义

＆＃13;

CREATE EXTERNAL TABLE meterevents ROW FORMAT SERDE org.apache.hadoop.hive.serde2.avro.AvroSerDe' STORED as INPUTFORMAT org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat' LOCATION '/......' TBLPROPERTIES ('avro.schema.url'='/..../schema.avsc');

＆＃13;

hive表结构

＆＃13;

nametype                struct<nametypedescription:string,nametypename:string,nametypeauthority:struct<nametypeauthorityname:string,nametypeauthoritydescription:string>>       from deserializer
names                   struct<name:string,nametype:struct<nametypedescription:string,nametypename:string,nametypeauthority:struct<nametypeauthorityname:string,nametypeauthoritydescription:string>>>  from deserializer
enddeviceeventdetails   struct<enddeviceeventdetailsname:string,enddeviceeventdetailsvalue:string>      from deserializer
enddeviceevent          struct<mrid:string,createddatetime:string,issuerid:string,issuertrackingid:string,reason:string,severity:string,userid:string,asset:struct<assetmrid:string,assetnames:array<struct<name:string,nametype:struct<nametypedescription:string,nametypename:string,nametypeauthority:struct<nametypeauthorityname:string,nametypeauthoritydescription:string>>>>>,enddeviceeventdetails:array<struct<enddeviceeventdetailsname:string,enddeviceeventdetailsvalue:string>>,enddeviceeventtype:string,enddeviceeventnames:array<struct<name:string,nametype:struct<nametypedescription:string,nametypename:string,nametypeauthority:struct<nametypeauthorityname:string,nametypeauthoritydescription:string>>>>,status:struct<statusdatetime:string,statusreason:string,statusremark:string,statusvalue:string>,usagepoint:struct<usagepointmrid:string,usagepointnames:array<struct<name:string,nametype:struct<nametypedescription:string,nametypename:string,nametypeauthority:struct<nametypeauthorityname:string,nametypeauthoritydescription:string>>>>>>       from deserializer
enddeviceeventtype      struct<enddeviceeventtypemrid:string,enddeviceeventtypedomain:string,enddeviceeventtypeeventoraction:string,enddeviceeventtypesubdomain:string,type:string,enddeviceeventtypenames:array<struct<name:string,nametype:struct<nametypedescription:string,nametypename:string,nametypeauthority:struct<nametypeauthorityname:string,nametypeauthoritydescription:string>>>>>       from deserializer
header                  struct<noun:string,context:string,verb:string,value:string,source:string,timestamp:string,correlationid:string,name:string,messageid:string,property:struct<propertyname:array<string>,propertyvalue:array<string>>>    from deserializer
payload                 struct<enddeviceevents:array<struct<mrid:string,createddatetime:string,issuerid:string,issuertrackingid:string,reason:string,severity:string,userid:string,asset:struct<assetmrid:string,assetnames:array<struct<name:string,nametype:struct<nametypedescription:string,nametypename:string,nametypeauthority:struct<nametypeauthorityname:string,nametypeauthoritydescription:string>>>>>,enddeviceeventdetails:array<struct<enddeviceeventdetailsname:string,enddeviceeventdetailsvalue:string>>,enddeviceeventtype:string,enddeviceeventnames:array<struct<name:string,nametype:struct<nametypedescription:string,nametypename:string,nametypeauthority:struct<nametypeauthorityname:string,nametypeauthoritydescription:string>>>>,status:struct<statusdatetime:string,statusreason:string,statusremark:string,statusvalue:string>,usagepoint:struct<usagepointmrid:string,usagepointnames:array<struct<name:string,nametype:struct<nametypedescription:string,nametypename:string,nametypeauthority:struct<nametypeauthorityname:string,nametypeauthoritydescription:string>>>>>>>,enddeviceeventtype:array<struct<enddeviceeventtypemrid:string,enddeviceeventtypedomain:string,enddeviceeventtypeeventoraction:string,enddeviceeventtypesubdomain:string,type:string,enddeviceeventtypenames:array<struct<name:string,nametype:struct<nametypedescription:string,nametypename:string,nametypeauthority:struct<nametypeauthorityname:string,nametypeauthoritydescription:string>>>>>>>

＆＃13;

我正在使用＆＃34; LATERAL VIEW爆炸＆＃34;我的查询中的选项

＆＃13;

select eddetails.enddeviceeventdetailsname,	eddetails.enddeviceeventdetailsvalue 
FROM meterevents_tmp 
LATERAL VIEW explode(payload.enddeviceevents.enddeviceeventdetails)  ed AS eddetails 
limit 1;

＆＃13;

但我仍然以单行获取数据。

＆＃13;

enddeviceeventdetailsname       enddeviceeventdetailsvalue
["EventSequenceNumber","EventSequenceNumber","EventSequenceNumber","EventSequenceNumber"]       ["683","684","685","686"

＆＃13;

我想将此数据作为

＆＃13;

enddeviceeventdetailsname       enddeviceeventdetailsvalue
EventSequenceNumber              683
EventSequenceNumber              684
EventSequenceNumber              685
EventSequenceNumber              686

＆＃13;

我已经在stackoverflow上阅读了另一个问题：Exploding Array of Struct using HiveQL

但无法获得预期的输出。因为在那个帖子中它的蜂巢外部表而不是我无法指定的serde＆＃34; MAP KEYS TERMINATED BY＆＃34;和＆＃34;收集项目由＆＃34;

终止

非常感谢任何帮助。

由于

Answer 1

我能够解决这个问题---

我无法在行中获取输出，因为

array<struct<string,string>>

是父数组的一部分

array<struct<array<struct<string, string>>>

我更新了我的查询并使用了嵌套的爆炸

select eddetails.enddeviceeventdetailsname, eddetails.enddeviceeventdetailsvalue from (select ede.enddeviceeventdetails FROM meterevents_tmp LATERAL VIEW explode(payload.enddeviceevents)  e AS ede) t LATERAL VIEW explode(t.enddeviceeventdetails)  ed AS eddetails limit 10;

我得到了所需的输出 -

enddeviceeventdetailsname       enddeviceeventdetailsvalue
EventSequenceNumber     683
EventSequenceNumber     684
EventSequenceNumber     685
EventSequenceNumber     686

无法获取数组<struct <string，string =“”>＆gt;的avro数据在行

1 个答案: