Question

我正在尝试将表从mysql导入到Sqoop。但是在运行sqoop导入时 - 该作业陷入了Mapreduce Job。

请帮忙

使用代码： sqoop import --connect jdbc:mysql://localhost/test --direct --username=root --password= --table=authors --hive-import --hive-table=mydb.authors --target-dir=user/root/sample -m 1

日志：

16/10/31 14:32:11 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/10/31 14:32:11 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
16/10/31 14:32:11 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
16/10/31 14:32:11 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
16/10/31 14:32:11 INFO tool.CodeGenTool: Beginning code generation
16/10/31 14:32:12 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `authors` AS t LIMIT 1
16/10/31 14:32:12 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `authors` AS t LIMIT 1
16/10/31 14:32:12 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
16/10/31 14:32:12 INFO orm.CompilationManager: Found hadoop core jar at: /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar
Note: /tmp/sqoop-root/compile/509cb57707137dd45538cf81fd7e11b1/authors.java uses or overrides a deprecated API.
16/10/31 14:32:11 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/10/31 14:32:11 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
16/10/31 14:32:11 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
16/10/31 14:32:11 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
16/10/31 14:32:11 INFO tool.CodeGenTool: Beginning code generation
16/10/31 14:32:12 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `authors` AS t LIMIT 1
16/10/31 14:32:12 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `authors` AS t LIMIT 1
16/10/31 14:32:12 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
16/10/31 14:32:12 INFO orm.CompilationManager: Found hadoop core jar at: /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar
Note: /tmp/sqoop-root/compile/509cb57707137dd45538cf81fd7e11b1/authors.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
16/10/31 14:32:14 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/509cb57707137dd45538cf81fd7e11b1/authors.jar
16/10/31 14:32:14 INFO manager.DirectMySQLManager: Beginning mysqldump fast path import
16/10/31 14:32:14 INFO mapreduce.ImportJobBase: Beginning import of authors

Answer 1

你可以在执行sqoop之前创建一个hive表，如果你不使用--create-hive-table，你的hive表是否存在？

例如：

sqoop import 
--connect jdbc:oracle:thin:@//***/orcl 
--username ***
--password ***
--table TEST
--hive-import 
--create-hive-table 
--hive-database myDB 
--hive-table table1
--hive-override

sqoop导入在运行MAPREDUCE作业时挂起

1 个答案: