我有一个PigLatin代码,它使用以下模式生成对象A:
pages: {timestamp: datetime,url: chararray,source_url: chararray,status_code: chararray,http_headers: map[],content: chararray}
然后我使用(使用-schema
选项)存储它:
STORE pages INTO 'pages' USING PigStorage('\t', '-schema');
当我将其加载回来时(从另一个脚本),我正确地看到了描述:
pages = LOAD 'data/layers/raw_content';
DESCRIBE pages;
> pages: {timestamp: datetime,url: chararray,source_url: chararray,status_code: chararray,http_headers: map[],content: chararray}
但是当我运行以下语句时,它失败了:
A = FOREACH pages GENERATE http_headers#'content-type';
DUMP A;
2014-07-13 14:49:44,398 [Thread-14] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local_0004
org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing [POMapLookUp (Name: POMapLookUp[bytearray] - scope-33 Operator Key: scope-33) children: null at [null[26,39]]]: java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.util.Map
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:338)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:378)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:298)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.util.Map
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:99)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNextDataByteArray(POMapLookUp.java:116)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:313)
... 9 more
完整错误日志位于:http://pastebin.com/BLzsciU8
如果我没有加载pages
,而是复制生成它的脚本,一切正常。
有人可以帮忙吗?