我使用以下sqoop命令将MySQL数据导入hive。
MySQL表:
create table student(id int,name varchar(20),rank int,state varchar(20),primary key(id));
id | name | rank | state |
+-----+------+------+-----------+
| 100 | AAA | 5 | AP |
| 101 | BBB | 2 | AP |
| 102 | CCC | 1 | Karnataka |
| 103 | DDD | 10 | Karnataka |
| 104 | EEE | 2 | AP |
| 105 | FFF | 3 | AP |
| 106 | GGGG | 4 | AP |
| 107 | HHH | 5 | AP |
| 108 | IIII | 15 | Karnataka |
| 109 | JJJJ | 8 | Karnataka
sqoop import --connect jdbc:mysql://localhost/test --username root --password hadoop --table student --hive-import --fields-terminated-by "," --lines-terminated-by "\n" --warehouse-dir /user/hive/warehouse/test.db
sqoop成功将数据导入test.db/student
目录。我为该地点创建了一个蜂巢学生表。
我在MySQL学生表中添加了一些行,如
id | name | rank | state |
+-----+------+------+-----------+
| 100 | AAA | 5 | AP |
| 101 | BBB | 2 | AP |
| 102 | CCC | 1 | Karnataka |
| 103 | DDD | 10 | Karnataka |
| 104 | EEE | 2 | AP |
| 105 | FFF | 3 | AP |
| 106 | GGGG | 4 | AP |
| 107 | HHH | 5 | AP |
| 108 | IIII | 15 | Karnataka |
| 109 | JJJJ | 8 | Karnataka |
| 110 | KKK | 6 | Karnataka |
| 111 | LLLL | 7 | AP |
| 112 | MMM | 9 | AP
为了导入新插入的行,我使用sqoop命令作为
sqoop import --connect jdbc:mysql://localhost/test --username root --password hadoop --table student --fields-terminated-by "," --lines-terminated-by "\n" --warehouse-dir /user/hive/warehouse/test.db --incremental append --check-column id --last-value 109
但是这个sqoop命令正在重启我的VM。有什么问题?我无法找到原因。
我为同一个命令创建了一个作业,
sqoop job --create import-student1 -- import --connect jdbc:mysql://localhost/test --username root --password hadoop --table student --incremental append --check-column id --last-value 109 --warehouse-dir /user/hive/warehouse/test.db --fields-terminated-by "," --lines-terminated-by "\n" --direct
sqoop job --exec import-student1
然后显示已成功完成,但数据未导入