mongodb集合未加载到配置单元表中

时间:2018-11-29 02:51:04

标签: mongodb hadoop hive

我目前正在尝试使用 hive 迁移 mongodb集合 hdfs

我成功创建了基于mongodb的配置单元表,当我浏览hdfs UI时,该表位于我的hdfs目录中。

但是仅显示表格,但表格为空无可用数据

似乎mongodb集合未加载到表中。下面是我如何创建表格。

create table mongo(
id String,
age String,
gender String,
race String,
custState String,
purchaseDate String,
purchaseTime String,
foodname String,
restaurant String,
foodtype String,
quantity String,
totalPrice String,
orderType String,
rating String,
servingType String,
characteristic String,
restaurantType String,
restaurantState String,
priceRange String,
paymentMethod String,
tableBooking String,
onlineBooking String,
deliveryService String)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
WITH SERDEPROPERTIES('mongo.columns.mapping'='{
"id":"_id",
"age":"age",
"gender":"gender",
"race":"race",
"custState":"custState ",
"purchaseDate":"purchaseDate",
"purchaseTime":"purchaseTime",
"foodname":"foodname",
"restaurant":"restaurant",
"foodtype":"foodtype",
"quantity":"quantity",
"totalPrice":"totalPrice",
"orderType":"orderType",
"rating":"rating",
"servingType":"servingType",
"characteristic":"characteristic",
"restaurantType":"restaurantType",
"restaurantState":"restaurantState",
"priceRange":"priceRange",
"paymentMethod":"paymentMethod",
"tableBooking":"tableBooking",
"onlineBooking":"onlineBooking",
"deliveryService":"deliveryService"}')
TBLPROPERTIES('mongo.uri'='mongodb://master:27017/bigdatadb.datasetv1');

在创建表之前,我还根据建议添加了jar,但我不确定它的意义是什么。

add jar /usr/local/hadoop/lib/hive-serde-1.0.jar;
add jar /usr/local/hadoop/lib/mongo-hadoop-core-1.5.2.jar;
add jar /usr/local/hadoop/lib/mongo-hadoop-hive-1.5.2.jar;
add jar /usr/local/hadoop/lib/mongo-java-driver-3.8.1.jar;
add jar /usr/local/Hive-JSON-Serde/json-serde/target/json-serde-1.3.9-SNAPSHOT-jar-with-dependencies.jar;

我正在使用hadoop 2.9.1,mongodb 4.0.4和hive 2.3.3。有人可以帮我解决此问题吗?

0 个答案:

没有答案