Question

我正在尝试将MongoDb与Hadoop连接。我在我的Ubuntu 14.04中安装了Hadoop-1.2.1。我安装了MongoDB-3.0.4，并在hive会话中下载并添加了mongo-hadoop-hive-1.3.0.jar，mongo-java-driver-2.13.2.jar个jar。我已下载mongo-connector.sh（在此site中找到）并将其包含在Hadoop_Home / lib下。

我设置了这样的输入和输出源：

hive> set MONGO_INPUT=mongodb://[user:password@]<MongoDB Instance IP>:27017/DBname.collectionName;
hive> set MONGO_OUTPUT=mongodb://[user:password@]<MongoDB Instance IP>:27017/DBname.collectionName;
hive> add JAR brickhouse-0.7.0.jar;
hive> create temporary function collect as 'brickhouse.udf.collect.CollectUDAF';

我在MongoDb的收藏是这样的：

> db.shows.find()
{ "_id" : ObjectId("559eb22fa7999b1a5f50e4e6"), "title" : "Arrested Development", "airdate" : "November 2, 2003", "network" : "FOX" }
{ "_id" : ObjectId("559eb238a7999b1a5f50e4e7"), "title" : "Stella", "airdate" : "June 28, 2005", "network" : "Comedy Central" }
{ "_id" : ObjectId("559eb23ca7999b1a5f50e4e8"), "title" : "Modern Family", "airdate" : "September 23, 2009", "network" : "ABC" }
>

现在我正在尝试创建一个Hive表

CREATE EXTERNAL TABLE mongoTest(title STRING,network STRING)
    > STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
    > WITH SERDEPROPERTIES('mongo.columns.mapping'='{"title":"name",”airdate”:”date”,”network”:”name”}')
    > TBLPROPERTIES('mongo.uri'='${hiveconf:MONGO_INPUT}');

当我运行此命令时，它会显示

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/util/JSON

然后我添加了hive-json-serde.jar和hive-serdes-1.0-SNAPSHOT.jar个jar，并尝试再次创建表格。但错误仍然相同。我该如何纠正这个错误？

Answer 1

我实际上在Hadoop_Home / lib文件夹中添加了这些mongo-hadoop-core-1.3.0.jar，mongo-hadoop-hive-1.3.0.jar和mongo-java-driver-2.13.2.jar个jar。然后我能够从MongoDb到Hive获取数据而没有任何错误。

Answer 2

解析器正在看到智能引号 - “

 ”airdate”:”date”,”network”:”name”

他们应该

 "airdate":"date","network":"name"

执行错误，从org.apache.hadoop.hive.ql.exec.DDLTask返回代码1。 COM /的MongoDB / UTIL / JSON

2 个答案: