Question

我有一个mysql表＆＃34;客户＆＃34;并尝试使用sqoop import将数据从mysql导入到hdfs位置。以下是我在我的机器上安装的版本： sqoop版本：1.4.6 蜂巢版：2.3.0 Hadoop版本：2.8.1

Sqoop import命令：

sqoop import --connect jdbc:mysql://localhost/localdb --username root --password mnbv@1234 --table customers -m 1 --target-dir /user/hduser/sqoop_import/customers1/

并显示在ClassNotFountException以下：

Tue Oct 31 09:57:21 IST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.17/10/31 09:57:21 INFO db.DBInputFormat: Using read commited transaction isolation             17/10/31 09:57:21 INFO mapred.MapTask: Processing split: 1=1 AND 1=1 
17/10/31 09:57:21 INFO mapred.LocalJobRunner: map task executor complete. 
17/10/31 09:57:22 WARN mapred.LocalJobRunner: job_local1437452057_0001java.lang.Exception: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class customers not found 
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class customers not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2216) 
at org.apache.sqoop.mapreduce.db.DBConfiguration.getInputClass(DBConfiguration.java:403)
at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.createDBRecordReader(DataDrivenDBInputFormat.java:237)
at org.apache.sqoop.mapreduce.db.DBInputFormat.createRecordReader(DBInputFormat.java:263)     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:515)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:758)     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270)     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 
Caused by: java.lang.ClassNotFoundException: Class customers not found     at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2122)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2214)
... 12 more
17/10/31 09:57:22 INFO mapreduce.Job: Job job_local1437452057_0001 running in uber mode : false  17/10/31 09:57:22 INFO mapreduce.Job:  map 0% reduce 0%
17/10/31 09:57:22 INFO mapreduce.Job: Job job_local1437452057_0001 failed with state FAILED due to: NA  17/10/31 09:57:22 INFO mapreduce.Job: Counters: 0
17/10/31 09:57:22 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead  17/10/31 09:57:22 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 4.105 seconds (0 bytes/sec)
17/10/31 09:57:22 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead  17/10/31 09:57:22 INFO mapreduce.ImportJobBase: Retrieved 0 records.
17/10/31 09:57:22 ERROR tool.ImportTool: Error during import: Import job failed!

但是当我尝试使用sqoop命令列出表时，它工作正常并显示customers表。 Sqoop命令：

sqoop list-tables --connect jdbc:mysql://localhost/localdb --username root --password mnbv@1234;

输出正确显示如下：

17/10/31 10:07:09 INFO manager.MySQLManager: Preparing to use a MySQL  streaming resultset.          
Tue Oct 31 10:07:09 IST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+,  5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not  using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true  and provide truststore for server certificate verification.                                                                                                customers

可能是什么问题，为什么从表中导出sqoop无法识别mysql中的表。请帮助我。

先谢谢。

Answer 1

使用 --bindir 选项并指向当前的工作目录。在--bindir选项下，给出$ SQOOP_HOME / lib的路径

很抱歉迟到的回复。根据最新评论更新答案。请检查以下内容：

在运行sqoop进程时需要检查jar文件 cmd：Writing jar file:/tmp/sqoop.../*.jar
使用导入命令：sqoop import -fs local -jt local -libjars /tmp/..../*.jar --connect "jdbc:mysql://example.com/sqoop" --username <>--password <>--table customers
验证生成的Java文件是否为sqoop导入。它应该有与正常导入的表格相同。 sqoop codegen --connect <""> --username <> --password <> --table customers

希望这有帮助！

sqoop import未识别显示ClassNotFountException错误的mysql表

1 个答案: