使用Elephantbird JsonLoader如果记录采用以下格式,我可以加载数据:
{"disknum":36,"disksum":136.401,"disk_rate":1872.0,"disk_lnum": 13}
但实际数据采用以下格式:(括在方括号内)
[{"disknum":36,"disksum":136.401,"disk_rate":1872.0,"disk_lnum": 13}]
当我尝试解析它时,它不会抛出错误,也不会提供任何有用的输出。它显示成功!读取0条记录,写入0条记录。
请建议如何使用方括号处理数据。
下面的是非方括号记录的语法:
register '/home/data/Desktop/elephantbird/elephant-bird-core-4.1.jar';
register '/home/gopal/Desktop/elephantbird/elephant-bird-hadoop-compat-4.1.jar';
register '/home/gopal/Desktop/elephantbird/elephant-bird-pig-4.1.jar';
register '/home/gopal/Desktop/elephantbird/json-simple-1.1.jar';
a = LOAD '/pig/tc1.log' USING com.twitter.elephantbird.pig.load.JsonLoader() as (json:map[]);
b = FOREACH a GENERATE flatten(json#'node_disk_lnum_1') AS node_disk_lnum_1, flatten(json#'node_disk_xfers_in_rate_sum') AS node_disk_xfers_in_rate_sum, flatten(json#'node_disk_bytes_in_rate_22') AS node_disk_bytes_in_rate_22, flatten(json#'node_disk_lnum_7') AS node_disk_lnum_7;
dump b;
请指教!在此先感谢:)