无法加载地图字段

时间:2014-07-13 11:56:22

标签: hadoop apache-pig

我有一个PigLatin代码,它使用以下模式生成对象A:

pages: {timestamp: datetime,url: chararray,source_url: chararray,status_code: chararray,http_headers: map[],content: chararray}

然后我使用(使用-schema选项)存储它:

STORE pages INTO 'pages' USING PigStorage('\t', '-schema');

当我将其加载回来时(从另一个脚本),我正确地看到了描述:

pages = LOAD 'data/layers/raw_content';
DESCRIBE pages;
> pages: {timestamp: datetime,url: chararray,source_url: chararray,status_code: chararray,http_headers: map[],content: chararray}

但是当我运行以下语句时,它失败了:

A = FOREACH pages GENERATE http_headers#'content-type';
DUMP A;

2014-07-13 14:49:44,398 [Thread-14] WARN  org.apache.hadoop.mapred.LocalJobRunner - job_local_0004
org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing     [POMapLookUp (Name: POMapLookUp[bytearray] - scope-33 Operator Key: scope-33) children: null     at [null[26,39]]]: java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.util.Map
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:338)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:378)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:298)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.util.Map
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:99)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNextDataByteArray(POMapLookUp.java:116)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:313)
... 9 more

完整错误日志位于:http://pastebin.com/BLzsciU8

如果我没有加载pages,而是复制生成它的脚本,一切正常。

有人可以帮忙吗?

0 个答案:

没有答案