使用avro架构编写时如何在拉丁猪中创建复杂记录

时间:2013-08-22 13:56:49

标签: hadoop apache-pig

我在拉丁猪中有以下关系,我需要将它存储到avro-schema文件中:

    data = load 'data.dat' as (a: long, b: chararray, c: long);
...
    STORE data INTO 'file' USING avroStorage();

但我需要数据格式为:

Record 1:
{
  "a" : {
    "long" : 2
  },
  "complex" : {
      "b" : {
        "string" : "abc"
      },
      "c" : {
        "long" : 3
      }
   }
}
...

1 个答案:

答案 0 :(得分:0)

我没有测试过这段代码,看看它是如何转换为avro架构但是尝试从这段代码开始

data = load 'data.dat' as (a: long, b: chararray, c: long);    
transformed = foreach data generate a, TOTUPLE(b, c);        
STORE transformed INTO 'file' USING avroStorage();

TOTUPLE的文档 - http://pig.apache.org/docs/r0.11.1/func.html#totuple