SSIS-通过ODBC批量加载Hive表

时间:2018-12-18 20:17:43

标签: ssis hive odbc

我正在尝试将SSIS包移植到针对HIVE的目标而不是DB2。我使用Hortonworks HIVE ODBC遇到ODBC Destination任务时遇到问题。我以前尝试过Microsoft ODBC,但未能创建有效的插入查询。

我阅读了文档“从SQL Server Integration Services中利用Hadoop群集”,其中提到不可能从ODBC写入Hive,但这是2013年的文档。

问题: 我有一个带有OLE DB源(DB2)和ODBC目标(Hive)的数据流任务。如果我将ODBC目标设置为“逐行”,则加载将按预期完成。但是,如果我将其设置为“批量”,那么我只会收到以下错误:

错误:将令牌推送到HIVE时出现0x384,ODBC目标[2]:发生打开数据库连接(ODBC)错误。插入第1行时,SQLExecute返回错误

我已附上有问题的任务的ODBC跟踪。我希望在跟踪中看到一个SQLExecute SQL_ERROR,但似乎丢失了。

是否甚至可以通过ODBC来实现?还是需要在Hive Task之后使用HDFS Destination?

DtsDebugHost    1e9c-2488   ENTER SQLPrepareW 
        HSTMT               0x08D45108
        WCHAR *             0x0583CEE4 [      86] "INSERT INTO `default`.`temp_000000000000_return` ( `link_id`,`tok_data`) VALUES (?, ?)"
        SDWORD                    86

DtsDebugHost    1e9c-2488   EXIT  SQLPrepareW  with return code 0 (SQL_SUCCESS)
        HSTMT               0x08D45108
        WCHAR *             0x0583CEE4 [      86] "INSERT INTO `default`.`temp_000000000000_return` ( `link_id`,`tok_data`) VALUES (?, ?)"
        SDWORD                    86

DtsDebugHost    1e9c-2488   ENTER SQLSetStmtAttrW 
        SQLHSTMT            0x08D45108
        SQLINTEGER                  21 <SQL_ATTR_PARAMS_PROCESSED_PTR>
        SQLPOINTER          0x057D8578
        SQLINTEGER                   0 

DtsDebugHost    1e9c-2488   EXIT  SQLSetStmtAttrW  with return code 0 (SQL_SUCCESS)
        SQLHSTMT            0x08D45108
        SQLINTEGER                  21 <SQL_ATTR_PARAMS_PROCESSED_PTR>
        SQLPOINTER          0x057D8578
        SQLINTEGER                   0 

DtsDebugHost    1e9c-2488   ENTER SQLSetStmtAttrW 
        SQLHSTMT            0x08D45108
        SQLINTEGER                  20 <SQL_ATTR_PARAM_STATUS_PTR>
        SQLPOINTER          0x084B4158
        SQLINTEGER                   0 

DtsDebugHost    1e9c-2488   EXIT  SQLSetStmtAttrW  with return code 0 (SQL_SUCCESS)
        SQLHSTMT            0x08D45108
        SQLINTEGER                  20 <SQL_ATTR_PARAM_STATUS_PTR>
        SQLPOINTER          0x084B4158
        SQLINTEGER                   0 

DtsDebugHost    1e9c-2488   ENTER SQLBindParameter 
        HSTMT               0x08D45108
        UWORD                        1 
        SWORD                        1 <SQL_PARAM_INPUT>
        SWORD                      -16 <SQL_C_SLONG>
        SWORD                        4 <SQL_INTEGER>
        SQLULEN                   10
        SWORD                        0 
        PTR                0x08316178
        SQLLEN                     4
        SQLLEN *            0x08315F98

DtsDebugHost    1e9c-2488   EXIT  SQLBindParameter  with return code 0 (SQL_SUCCESS)
        HSTMT               0x08D45108
        UWORD                        1 
        SWORD                        1 <SQL_PARAM_INPUT>
        SWORD                      -16 <SQL_C_SLONG>
        SWORD                        4 <SQL_INTEGER>
        SQLULEN                   10
        SWORD                        0 
        PTR                0x08316178
        SQLLEN                     4
        SQLLEN *            0x08315F98 (137452312)

DtsDebugHost    1e9c-2488   ENTER SQLBindParameter 
        HSTMT               0x08D45108
        UWORD                        2 
        SWORD                        1 <SQL_PARAM_INPUT>
        SWORD                       -8 <SQL_C_WCHAR>
        SWORD                       -8 <SQL_WCHAR>
        SQLULEN                   35
        SWORD                        0 
        PTR                0x057F85C8
        SQLLEN                    72
        SQLLEN *            0x08315FB8

DtsDebugHost    1e9c-2488   EXIT  SQLBindParameter  with return code 0 (SQL_SUCCESS)
        HSTMT               0x08D45108
        UWORD                        2 
        SWORD                        1 <SQL_PARAM_INPUT>
        SWORD                       -8 <SQL_C_WCHAR>
        SWORD                       -8 <SQL_WCHAR>
        SQLULEN                   35
        SWORD                        0 
        PTR                0x057F85C8
        SQLLEN                    72
        SQLLEN *            0x08315FB8 (137452312)

DtsDebugHost    1e9c-2c5c   ENTER SQLExecute 
        HSTMT               0x08D45108

DtsDebugHost    1e9c-2c5c   EXIT  SQLExecute  with return code 0 (SQL_SUCCESS)
        HSTMT               0x08D45108

DtsDebugHost    1e9c-2c5c   ENTER SQLGetDiagRecW 
        SQLSMALLINT                  3 <SQL_HANDLE_STMT>
        SQLHANDLE           0x08D45108
        SQLSMALLINT                  1 
        SQLWCHAR *          0x0927EFA0
        SQLINTEGER *        0x0927D784
        SQLWCHAR *          0x0927DF98 
        SQLSMALLINT               1024 
        SQLSMALLINT *       0x0927D788

DtsDebugHost    1e9c-2c5c   EXIT  SQLGetDiagRecW  with return code 100 (SQL_NO_DATA_FOUND)
        SQLSMALLINT                  3 <SQL_HANDLE_STMT>
        SQLHANDLE           0x08D45108
        SQLSMALLINT                  1 
        SQLWCHAR *          0x0927EFA0
        SQLINTEGER *        0x0927D784
        SQLWCHAR *          0x0927DF98 
        SQLSMALLINT               1024 
        SQLSMALLINT *       0x0927D788

DtsDebugHost    1e9c-2488   ENTER SQLFreeHandle 
        SQLSMALLINT                  3 <SQL_HANDLE_STMT>
        SQLHANDLE           0x08D45108

DtsDebugHost    1e9c-2488   EXIT  SQLFreeHandle  with return code 0 (SQL_SUCCESS)
        SQLSMALLINT                  3 <SQL_HANDLE_STMT>
        SQLHANDLE           0x08D45108

DtsDebugHost    1e9c-2488   ENTER SQLDisconnect 
        HDBC                0x0254EBD8

DtsDebugHost    1e9c-2488   EXIT  SQLDisconnect  with return code 0 (SQL_SUCCESS)
        HDBC                0x0254EBD8

DtsDebugHost    1e9c-2488   ENTER SQLFreeHandle 
        SQLSMALLINT                  2 <SQL_HANDLE_DBC>
        SQLHANDLE           0x0254EBD8

DtsDebugHost    1e9c-2488   EXIT  SQLFreeHandle  with return code 0 (SQL_SUCCESS)
        SQLSMALLINT                  2 <SQL_HANDLE_DBC>
        SQLHANDLE           0x0254EBD8

DtsDebugHost    1e9c-2488   ENTER SQLFreeHandle 
        SQLSMALLINT                  1 <SQL_HANDLE_ENV>
        SQLHANDLE           0x0254E748

DtsDebugHost    1e9c-2488   EXIT  SQLFreeHandle  with return code 0 (SQL_SUCCESS)
        SQLSMALLINT                  1 <SQL_HANDLE_ENV>
        SQLHANDLE           0x0254E748

0 个答案:

没有答案