Question

如何使用json架构插入覆盖目录？

有原始的蜂巢avro表; （这实际上有很多领域）

tb_test--------
name string
kickname string
-----------------

然后我想通过jsonserde将查询结果保存到hdfs中的某个目录中。

我试过了。

insert overwrite directory '/json/'
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
WITH SERDEPROPERTIES (
 "stat_name"="$._col0",
 "stat_interval"="$._col1"
)
STORED AS TEXTFILE 
select name, nickname
from tb_test limit 100

但是/ json /中写的json有_colXX字段名而不是原始字段名。

{"_col0":"basic_qv"," _col1":"h"}
{"_col0":"basic_qv","_col1 ":"h"}
{"_col0":"basic_qv","_col1 ":"h"}
{"_col0":"basic_qv"," _col1":"h"}
{"_col0":"basic_qv","_col1 ":"h"}

我期待

{"name":"basic_qv","nickname":"h"}
{"name":"basic_qv","nickname":"h"}
{"name":"basic_qv","nickname":"h"}
{"name":"basic_qv","nickname":"h"}
{"name":"basic_qv","nickname":"h"}

对此有何帮助？

谢谢！

Answer 1

这里描述了一个解决方法的问题（使用带有named_struct的JsonUDF）：https://github.com/rcongiu/Hive-JSON-Serde/issues/151

Completed 500 Internal Server Error in 18ms (ActiveRecord: 1.1ms)



ActionView::Template::Error (undefined method `map' for "chocolate, citrus":String
Did you mean?  tap):
    39:
    40: <p>
    41:   <strong>Tasting Notes</strong>
    42:   <%= raw @roast.tastingnotes.map(&:name).map { |t| link_to t, tastingnotes_path(t) }.join(', ') %>
    43: </p>
    44:
    45:

您还可以创建基于JsonSerDe的表，并定义列extract.hql: add jar /home/myuser/lib/json-udf-1.3.8-SNAPSHOT-jar-with-dependencies.jar; create temporary function tjson as 'org.openx.data.udf.JsonUDF'; insert overwrite local directory '/json/' select tjson(named_struct("name", t.name,"nickname", t.nickname)) from tb_test t ;，并使用表位置而不是目录。

HIVE使用json格式插入覆盖目录

1 个答案: