Question

我有存储在hdfs中的数据，数据前后有空格，当我尝试导出到mysql时，它会给出numberformat异常但是当我创建没有空格的数据时，它已成功插入到mysql中。

my question is can't we export the data which has space from hdfs to mysql usong sqoop export command?

The data which i used
1201, adi, sen manager,   30000,  it
1201, pavan, jun manager, 5000,    cs
1203, santhosh, junior,    60000,  mech

i created table like
create table emp(id BIGINT,name varchar(20),desg varchar(20),salary BIGINT,dept varchar(20));

sqoop command -- sqoop export \
--connect jdbc:mysql://127.0.0.1/mydb \
--username root \
--table emp \
--m 1 \
--export-dir /mydir \
--input-fields-terminated-by ',' \
--input-lines-terminated-by '\n'

result: numberformatexception input string:'1201'
can't parse the data

i discussed in forum, they said trim the space but i wants to know that automatically trim the spaces while perform sqoop export.

can somebody give suggestions on this?

Answer 1

你可以做一件简单的事情：

使用所有VARCHAR

在MySQL中创建临时表

create table emp-temp(id BIGINT,name varchar(20),desg varchar(20),salary BIGINT,dept varchar(20));

现在在TRIM()和CAST()

之后创建另一个包含数字字段的字段

create table emp as select CAST(TRIM(id) AS UNSIGNED), name, desg, CAST(TRIM(salary) AS UNSIGNED), dept FROM emp_temp;

Answer 2

Sqoop 在内部运行MapReduce作业。

简单的解决方案是运行映射器并修剪数据中的空格，并将输出输出到不同的文件中并运行 sqoop export 新文件。

sqoop导出命令，用于在hdfs之前有空格的数据

2 个答案: