我正在尝试使用JAVA运行一个简单的sqoop导入程序。
我的计划：
String driver="com.vertica.Driver";
Configuration config = new Configuration();
config.addResource(new Path("/../../../mapred-site.xml"));
config.addResource(new Path("/../../../core-site.xml"));
config.addResource(new Path("/../../../hdfs-site.xml"));
SqoopOptions options = new SqoopOptions();
options.setConnectString(connection_string);
options.setDriverClassName(driver);
options.setUsername(username);
options.setPassword("xxxxxxxxxxxxx");
options.setTargetDir(directory path);
options.setFetchSize(0);
options.setSqlQuery("SELECT 123456 as runid, " +
"business_date " +
"FROM database.table " +
"WHERE business_date=\'20140922\' " +
"AND instrument_id=1037894110378941 " +
"AND $CONDITIONS");
int ret = new ImportTool().run(options);
System.out.println("Return Val: "+ret);
我有以下错误
15/02/03 02:26:23 WARN sqoop.ConnFactory: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
15/02/03 02:26:23 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
15/02/03 02:26:23 INFO tool.CodeGenTool: Beginning code generation
15/02/03 02:26:24 INFO manager.SqlManager: Executing SQL statement: SELECT 123456 as runid, business_date FROM silverscreen_qa.SS_INSTRUMENT WHERE business_date='20140922' AND instrument_id=1037894110378941 AND (1 = 0)
15/02/03 02:26:24 INFO manager.SqlManager: Executing SQL statement: SELECT 123456 as runid, business_date FROM silverscreen_qa.SS_INSTRUMENT WHERE business_date='20140922' AND instrument_id=1037894110378941 AND (1 = 0)
15/02/03 02:26:24 INFO orm.CompilationManager: $HADOOP_MAPRED_HOME is not set
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:11: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.JdbcWritableBridge;
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:12: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.DelimiterSet;
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:13: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.FieldFormatter;
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:14: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.RecordParser;
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:15: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.BooleanParser;
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:16: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.BlobRef;
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:17: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.ClobRef;
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:18: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.LargeObjectLoader;
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:19: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.SqoopRecord;
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:37: error: cannot find symbol
public class QueryResult extends SqoopRecord implements DBWritable, Writable {
^
symbol: class SqoopRecord
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:81: error: cannot find symbol
public void loadLargeObjects(LargeObjectLoader __loader)
^
symbol: class LargeObjectLoader
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:119: error: cannot find symbol
private final DelimiterSet __outputDelimiters = new DelimiterSet((char) 44, (char) 10, (char) 0, (char) 0, false);
^
symbol: class DelimiterSet
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:123: error: cannot find symbol
public String toString(DelimiterSet delimiters) {
^
symbol: class DelimiterSet
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:129: error: cannot find symbol
public String toString(DelimiterSet delimiters, boolean useRecordDelim) {
^
symbol: class DelimiterSet
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:140: error: cannot find symbol
private final DelimiterSet __inputDelimiters = new DelimiterSet((char) 44, (char) 10, (char) 0, (char) 0, false);
^
symbol: class DelimiterSet
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:141: error: cannot find symbol
private RecordParser __parser;
^
symbol: class RecordParser
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:142: error: package RecordParser does not exist
public void parse(Text __record) throws RecordParser.ParseError {
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:150: error: package RecordParser does not exist
public void parse(CharSequence __record) throws RecordParser.ParseError {
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:158: error: package RecordParser does not exist
public void parse(byte [] __record) throws RecordParser.ParseError {
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:166: error: package RecordParser does not exist
public void parse(char [] __record) throws RecordParser.ParseError {
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:174: error: package RecordParser does not exist
public void parse(ByteBuffer __record) throws RecordParser.ParseError {
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:182: error: package RecordParser does not exist
public void parse(CharBuffer __record) throws RecordParser.ParseError {
^
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:64: error: bad operand types for binary operator '=='
if (this == o) {
^
first type: QueryResult
second type: Object
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:67: error: inconvertible types
if (!(o instanceof QueryResult)) {
^
required: QueryResult
found: Object
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:70: error: inconvertible types
QueryResult that = (QueryResult) o;
^
required: QueryResult
found: Object
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:78: error: cannot find symbol
this.runid = JdbcWritableBridge.readLong(1, __dbResults);
^
symbol: variable JdbcWritableBridge
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:79: error: cannot find symbol
this.business_date = JdbcWritableBridge.readDate(2, __dbResults);
^
symbol: variable JdbcWritableBridge
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:89: error: cannot find symbol
JdbcWritableBridge.writeLong(runid, 1 + __off, -5, __dbStmt);
^
symbol: variable JdbcWritableBridge
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:90: error: cannot find symbol
JdbcWritableBridge.writeDate(business_date, 2 + __off, 91, __dbStmt);
^
symbol: variable JdbcWritableBridge
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:119: error: cannot find symbol
private final DelimiterSet __outputDelimiters = new DelimiterSet((char) 44, (char) 10, (char) 0, (char) 0, false);
^
symbol: class DelimiterSet
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:132: error: cannot find symbol
__sb.append(FieldFormatter.escapeAndEnclose(runid==null?"null":"" + runid, delimiters));
^
symbol: variable FieldFormatter
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:134: error: cannot find symbol
__sb.append(FieldFormatter.escapeAndEnclose(business_date==null?"null":"" + business_date, delimiters));
^
symbol: variable FieldFormatter
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:140: error: cannot find symbol
private final DelimiterSet __inputDelimiters = new DelimiterSet((char) 44, (char) 10, (char) 0, (char) 0, false);
^
symbol: class DelimiterSet
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:144: error: cannot find symbol
this.__parser = new RecordParser(__inputDelimiters);
^
symbol: class RecordParser
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:152: error: cannot find symbol
this.__parser = new RecordParser(__inputDelimiters);
^
symbol: class RecordParser
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:160: error: cannot find symbol
this.__parser = new RecordParser(__inputDelimiters);
^
symbol: class RecordParser
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:168: error: cannot find symbol
this.__parser = new RecordParser(__inputDelimiters);
^
symbol: class RecordParser
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:176: error: cannot find symbol
this.__parser = new RecordParser(__inputDelimiters);
^
symbol: class RecordParser
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:184: error: cannot find symbol
this.__parser = new RecordParser(__inputDelimiters);
^
symbol: class RecordParser
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:206: error: cannot find symbol
QueryResult o = (QueryResult) super.clone();
^
symbol: variable super
location: class QueryResult
/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:208: error: incompatible types
return o;
^
required: Object
found: QueryResult
41 errors
15/02/03 02:26:24 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Error returned by javac
at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:207)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:97)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
at com.cts.rbc.archive.TestConnectionForHive.main(TestConnectionForHive.java:196)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Return Val: 1
我尝试过可能的解决方案。
在我的hadoop-env.sh中，JAVA_HOME的值是
JAVA_HOME=/usr/lib/jvm/java-1.7.0-oracle-1.7.0.65.x86_64
我知道这听起来很重复，但请相信我，我已经尝试过在类似问题中提供的解决方案中描述的所有解决方案
我尝试使用以下代码
更改〜/ .bashrc中的JAVA_HOME JAVA_HOME=/etc/alternatives/java
这是我的节点中安装最新java的地方 它仍然会引发同样的错误。
同样的查询在bash中有效。
注意：我无权将hadoop-env.sh中的JAVA_HOME更改为共享系统。
在Java程序中覆盖JAVA_HOME的可能方法？
答案 0 :(得分：0)
[解决]
无需更改java_home（或）hadoop_mapred_home（或）sqoop_home
如错误所示，它无法找到sqoop库。我在我的程序中使用了sqoop 1.4.4库。 请注意，我没有使用任何构建工具（Maven，SBT或任何东西） 我正在使用普通的eclipse并在我的程序中添加外部.jar。
导出.jar时我使用的是Eclipse标准方法（文件 - ＆gt;导出 - ＆gt; Java - ＆gt; Runnable .jar文件）
在启动配置中，我选择了适当的项目 在库处理中，我使用了“将所需的包打包到生成的JAR中”选项
当我选择“将所需的库提取到生成的JAR中”并执行.jar文件时，它已成功执行。