Question

我正在将一堆Oracle 10g表平移到hive中。我正在我的集群上开发Hortonworks HDP2.3。其中一个包含100多列的oracle表有一个ROW_ID列，我发现它的类型是oracle.sql.ROWID。 Sqoop投掷错误：

2016-02-04 16:09:19,746 ERROR - [main:] ~ Cannot resolve SQL type -8 (ClassWriter:645)
2016-02-04 16:09:19,747 ERROR - [main:] ~ Cannot resolve SQL type -8 (ClassWriter:645)
2016-02-04 16:09:19,747 ERROR - [main:] ~ No Java type for SQL type -8 for column ROW_ID (ClassWriter:718)
2016-02-04 16:09:19,748 ERROR - [main:] ~ No Java type for SQL type -8 for column ROW_ID (ClassWriter:718)
2016-02-04 16:09:19,749 ERROR - [main:] ~ No Java type for SQL type -8 for column ROW_ID (ClassWriter:798)
2016-02-04 16:09:19,756 ERROR - [main:] ~ Got exception running Sqoop: java.lang.NullPointerException (Sqoop:181)
java.lang.NullPointerException
        at org.apache.sqoop.orm.ClassWriter.parseNullVal(ClassWriter.java:1377)
        at org.apache.sqoop.orm.ClassWriter.parseColumn(ClassWriter.java:1402)
        at org.apache.sqoop.orm.ClassWriter.myGenerateParser(ClassWriter.java:1528)
        at org.apache.sqoop.orm.ClassWriter.generateParser(ClassWriter.java:1491)
        at org.apache.sqoop.orm.ClassWriter.generateClassForColumns(ClassWriter.java:1920)
        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1736)
        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

有没有办法将此oracle列类型映射到任何hive列类型或只是将其设置为字符串？

Answer 1

好的，不确定这是否是正确的方法。但无论如何。

首先我尝试--map-column-hive ROW_ID=String，它仍然抱怨同样的错误。

然后我尝试了--map-column-java ROW_ID=String，它很高兴地从oracle下载了数据，但还没准备好把它放进hive中：

2016-02-04 16:55:17,655 ERROR - [main:] ~ Encountered IOException running import job: java.io.IOException: Hive does not support the SQL type for column ROW_ID
        at org.apache.sqoop.hive.TableDefWriter.getCreateTableStmt(TableDefWriter.java:181)
        at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:188)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:514)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
 (ImportTool:613)

好吧，最后他们的组合工作得很好。

sqoop import .... --map-column-java ROW_ID=String --map-column-hive ROW_ID=String

此外，--split-by也不喜欢ROW_ID。所以我不得不使用不同的专栏。

-Cheers。

Sqoop无法将oracle列类型映射到hive列

1 个答案: