Question

我正在尝试将hdinsight群集中的hive表导出到Azure SQL。当我在两个表中都有一列时，我能够这样做，但如果列数更大，我有上面提到的异常。我已经尝试过不同的分隔符组合，没有任何帮助。

我已经创建了hive表，如下所示：

create table test2
(
a string, 
b string
)
row format delimited fields terminated by ',';

SQL表具有以下架构：

create table test1 
(
    a [nvarchar](100),
    b [nvarchar](100)
);

create clustered index test1_clustered_index on test1(a);

我使用以下脚本导出数据：

$tableName = 'test1'

$connectionString = "jdbc:sqlserver://$sqlDatabaseServerName.database.windows.net;user=$sqlDatabaseLogin@$sqlDatabaseServerName;password=$sqlDatabasePassword;database=$databaseName"

$exportDir = "/hive/warehouse/test2"

$sqoopDef = New-AzureHDInsightSqoopJobDefinition -Command "export --connect $connectionString --table $tableName --export-dir $exportDir --fields-terminated-by ','"

$sqoopJob = Start-AzureHDInsightJob -Cluster $clusterName -JobDefinition $sqoopDef

作业日志如下所示：

15/01/07 00:02:46 INFO mapreduce.Job: Task Id : attempt_1420540952382_0059_m_000000_1, Status : FAILED
Error: java.io.IOException: Can't export data, please check failed map task logs
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.RuntimeException: Can't parse input data: 'a,b'
    at test1.__loadFromFields(test1.java:204)
    at test1.parse(test1.java:147)
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
    ... 10 more
Caused by: java.util.NoSuchElementException
    at java.util.ArrayList$Itr.next(ArrayList.java:834)
    at test1.__loadFromFields(test1.java:199)
    ... 12 more

我做错了什么？提前致谢

将数据从Hive导出到SQL时出现Sqoop NoSuchElementException

0 个答案: