Question

我一直在Mongo DB中使用JavaScript编写的Map Reduce函数。现在由于大数据，我必须切换到Hadoop和Hive。

所以我需要用JavaScript函数实现HiveQL作为map并减少脚本。

我已经阅读了很多文档，说明脚本没有语言限制。供参考： - http://dev.bizo.com/2009/07/custom-map-scripts-and-hive.html

当我尝试使用这个简单的JavaScript函数作为地图脚本时使用HiveQL

function map(data){
    return data;
}

我收到此错误

*Error: java.lang.RuntimeException: Hive Runtime Error while closing operators
    at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:232)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: [Error 20003]: An error occurred when trying to close the Operator running your custom script.
    at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:550)
    at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
    at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
    at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
    at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:205)
    ... 8 more*

使用Hive中的Javascript映射Reduce

0 个答案: