Question

我已尝试使用tdimport实用程序将数据从teradata复制到hadoop但它失败了＆＃34; 15/12/11 07:21:28 INFO processor.HiveOutputProcessor：hive table default.pos_rtl_str_test不存在 ＆＃34;

如何在此处指定hive架构名称？

＆＃13;

[biadmin@ehaasp-10035-master-3]/usr/iop/4.1.0.0/hive/bin>/usr/iop/4.1.0.0/sqoop/bin/sqoop tdimport --connect jdbc:teradata://<<ipaddress>>/database=EDW01_V_LV_BASE --username <<username>> --password <<password>> --as-textfile --hive-table pos_rtl_str_test --table pos_rtl_str --columns "RTL_STR_ID, RTL_STR_LANG_CD" --split-by RTL_STR_ID
Warning: /usr/iop/4.1.0.0/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
15/12/11 07:20:47 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6_IBM_20
15/12/11 07:20:47 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
15/12/11 07:20:47 INFO common.ConnectorPlugin: load plugins in jar:file:/usr/iop/4.1.0.0/sqoop/lib/teradata-connector-1.4.1.jar!/teradata.connector.plugins.xml
15/12/11 07:20:47 WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
15/12/11 07:20:47 INFO hive.metastore: Trying to connect to metastore with URI thrift://<<masternode dns name>>:9083
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/iop/4.1.0.0/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/iop/4.1.0.0/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
15/12/11 07:20:48 INFO hive.metastore: Connected to metastore.
15/12/11 07:20:48 INFO processor.TeradataInputProcessor: input preprocessor com.teradata.connector.teradata.processor.TeradataSplitByHashProcessor starts at:  1449818448723
15/12/11 07:20:49 INFO utils.TeradataUtils: the input database product is Teradata
15/12/11 07:20:49 INFO utils.TeradataUtils: the input database version is 14.10
15/12/11 07:20:49 INFO utils.TeradataUtils: the jdbc driver version is 15.0
15/12/11 07:21:07 INFO processor.TeradataInputProcessor: the teradata connector for hadoop version is: 1.4.1
15/12/11 07:21:07 INFO processor.TeradataInputProcessor: input jdbc properties are jdbc:teradata://<<ipaddress>>/database=<<database>>
15/12/11 07:21:27 INFO processor.TeradataInputProcessor: the number of mappers are 4
15/12/11 07:21:27 INFO processor.TeradataInputProcessor: input preprocessor com.teradata.connector.teradata.processor.TeradataSplitByHashProcessor ends at:  1449818487899
15/12/11 07:21:27 INFO processor.TeradataInputProcessor: the total elapsed time of input preprocessor com.teradata.connector.teradata.processor.TeradataSplitByHashProcessor is: 39s
15/12/11 07:21:28 WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
15/12/11 07:21:28 INFO hive.metastore: Trying to connect to metastore with URI thrift://ehaasp-10035-master-3.bi.services.bluemix.net:9083
15/12/11 07:21:28 INFO hive.metastore: Connected to metastore.
15/12/11 07:21:28 INFO processor.HiveOutputProcessor: hive table default.pos_rtl_str_test does not exist
15/12/11 07:21:28 WARN tool.ConnectorJobRunner: com.teradata.connector.common.exception.ConnectorException: The output post processor returns 1
15/12/11 07:21:28 INFO processor.TeradataInputProcessor: input postprocessor com.teradata.connector.teradata.processor.TeradataSplitByHashProcessor starts at:  1449818488581
15/12/11 07:21:28 INFO processor.TeradataInputProcessor: input postprocessor com.teradata.connector.teradata.processor.TeradataSplitByHashProcessor ends at:  1449818488581
15/12/11 07:21:28 INFO processor.TeradataInputProcessor: the total elapsed time of input postprocessor com.teradata.connector.teradata.processor.TeradataSplitByHashProcessor is: 0s
15/12/11 07:21:28 ERROR wrapper.TDImportTool: Teradata Connector for Hadoop tool error.
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at com.ibm.biginsights.ie.sqoop.td.wrapper.TDImportTool.callTDCH(TDImportTool.java:104)
        at com.ibm.biginsights.ie.sqoop.td.wrapper.TDImportTool.run(TDImportTool.java:72)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: com.teradata.connector.common.exception.ConnectorException: Import Hive table's column schema is missing
        at com.teradata.connector.common.tool.ConnectorJobRunner.runJob(ConnectorJobRunner.java:140)
        ... 12 more

＆＃13;

Answer 1

您是否尝试完全限定蜂巢表的位置？该错误表明default数据库不包含表的架构。

tdimport --connect jdbc:teradata://<<ipaddress>>/database=EDW01_V_LV_BASE 
         --username <<username>> --password <<password>> 
         --as-textfile --hive-table {hivedb}.pos_rtl_str_test 
         --table pos_rtl_str --columns "RTL_STR_ID, RTL_STR_LANG_CD" 
         --split-by RTL_STR_ID

如果该表不存在且您尝试同时创建该表，则需要包含--map-column-hive参数。

tdimport实用程序无法将数据从teradata导入hadoop

1 个答案: