上次修改模式下的sqoop增量导入:错误工具。ImportTool:导入失败:java.io.IOException:无法加载jar

时间:2019-06-14 10:59:45

标签: java mysql ubuntu-16.04 sqoop

我已经使用sqoop将mySql数据库表导入了hdfs。该数据库可能每天都会更新,因此我可能需要在hdfs中反映这些更新。为此,我尝试将sqoop用作:

sqoop import --connect jdbc:mysql://127.0.0.1/crawl_data_stats --username root --password password --table scraped_metadata -m 1 --as-parquetfile --target-dir /databases/crawl_data_stats/scraped_metadata --check-column timestamp --incremental lastmodified --last-value '2019-05-16 13:06:31' --driver com.mysql.jdbc.Driver --merge-key id

这会产生错误:

ERROR tool.ImportTool: Import failed: java.io.IOException: Could not load jar /tmp/sqoop-root/compile/4bdd1f516ac0e2901cd52ff27ebd49c1/codegen_scraped_metadata.jar into JVM. (Could not find class scraped_metadata.)  
    at org.apache.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:92)  
    at com.cloudera.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:36)  
    at org.apache.sqoop.tool.ImportTool.loadJars(ImportTool.java:121)  
    at org.apache.sqoop.tool.ImportTool.lastModifiedMerge(ImportTool.java:463)  
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:529)  
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)  
    at org.apache.sqoop.Sqoop.run(Sqoop.java:147)  
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)  
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)  
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)  
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)  
    at org.apache.sqoop.Sqoop.main(Sqoop.java:252)  
Caused by: java.lang.ClassNotFoundException: scraped_metadata  
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)  
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)  
    at java.security.AccessController.doPrivileged(Native Method)  
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)  
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)  
    at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789)  
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)  
    at java.lang.Class.forName0(Native Method)  
    at java.lang.Class.forName(Class.java:266)  
    at org.apache.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:88)  
    ... 11 more  

I have just started using sqoop and don't know how to resolve this issue.
I came across the similar issue at  
https://stackoverflow.com/questions/56088078/error-tool-importtool-import-failed-java-io-ioexception-could-not-load-jar-t.  
However, there are no suggestions   

Thanks for help

0 个答案:

没有答案