Question

我发现很多人有类似的错误，但没有提示我的问题。

我的命令行：

sqoop import --connect jdbc:mysql://localhost/databaseY --username=root -P --table tableX --target-dir /user/ec2-user/databaseY/tableX --as-textfile --fields-terminated-by "\t"

错误

16/08/19 11:25:51 INFO mapreduce.Job: Job job_1471608424445_0028 running in uber mode : false
16/08/19 11:25:51 INFO mapreduce.Job:  map 0% reduce 0%
16/08/19 11:25:58 INFO mapreduce.Job:  map 25% reduce 0%
16/08/19 11:26:04 INFO mapreduce.Job:  map 50% reduce 0%
16/08/19 11:26:06 INFO mapreduce.Job: Task Id :     attempt_1471608424445_0028_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: java.sql.SQLException: Access denied for user 'root'@'localhost' (using password: YES)

由于此错误，地图如何开始和停止？

看起来我有所需的所有权利，因为这两个命令行正在运行：

sqoop list-databases --connect jdbc:mysql://localhost --username root -P

和

在具有root帐户的mysql上我可以做

select * from databaseY.tableX

--- --- EDIT

此命令行正在运行：

sqoop import --connect jdbc:mysql://localhost/databaseY --username root --password PASSWORD --query "select * from databaseY.tableX where number = 1474 AND \$CONDITIONS" --target-dir /tmp/ok --as-textfile --direct --split-by number

但是这个没有：

sqoop import --connect jdbc:mysql://localhost/databaseY --username root --password PASSWORD --query "select * from databaseY.tableX where 1 = 1 AND \$CONDITIONS" --target-dir /tmp/ok --as-textfile --direct --split-by number

然后，我意识到，如果我使用-m 1，我的sqoop导入工作正常。仅限 -m 1

这是否意味着我的群集配置错误？为什么我的工作只能用于一个地图任务？

-----解决方案-----

这只是IP地址的问题。我通过IP地址更改了localhost，现在工作正常。

Answer 1

Sqoop的工作方式。请参阅official doc。

如果某个表没有定义主键并且未提供--split-by，则导入将失败，除非使用--num-mappers 1选项将显式设置的数量显式设置为1或者 - 使用-autoreset-to-one-mapper选项。选项--autoreset-to-one-mapper通常与import-all-tables工具一起使用，以自动处理模式中没有主键的表。

Answer 2

此处的用户名和密码指定您尝试连接的数据库的用户和密码。这里的用户名和密码不适用于mysql登录用户。因此，请使用为您尝试连接的数据库指定的用户的用户名和密码。

Answer 3

迟到但并非最不重要，我想我将在这个帖子中给出答案部分

类似的问题我在面对sqoop时将表数据从主机windows机器导入到虚拟机客户机ubuntu os，只是因为我的windows主机安装了mysql服务器用户名并没有足够的权限来授予连接和模式远程请求。

所以，现在我没有谈论很多问题，我正在说明我面临的问题 -

vm4learning@vm4learning:~/Installations/sqoop-1.4.4$ bin/sqoop import --connect jdbc:mysql://192.168.56.1/india --table india_most_populated_cities --target-dir /user/vm4learning/remotedir/ --username root --password password -m 1
no main manifest attribute, in /home/vm4learning/Installations/hbase-0.94.14/lib/coprocessor.jar
find: paths must precede expression: sqoop-test-1.4.4.jar
Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]
Warning: $HADOOP_HOME is deprecated.

17/12/06 11:01:10 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
17/12/06 11:01:11 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
17/12/06 11:01:11 INFO tool.CodeGenTool: Beginning code generation
17/12/06 11:01:13 ERROR manager.SqlManager: Error executing statement: java.sql.SQLException: Access denied for user 'root'@'Administrator' (using password: YES)
java.sql.SQLException: Access denied for user 'root'@'Administrator' (using password: YES)

然后，我来到我的Windows主机，从cmd运行ipconfig并首先获取主机的ipaddress，在linux（ubuntu）中它是＆＃34; ifconfig＆＃34;：

Ethernet adapter VirtualBox Host-Only Network:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::54d4:4f16:4bdb:885%18
   IPv4 Address. . . . . . . . . . . : 192.168.56.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

所以这里我的IP地址是Windows主机上的192.168.56.1然后我通过Windows主机中的命令登录到mysql并在所述脚本下面运行以授予权限 -

mysql -u root -p 输入密码：********

的MySQL＆GT;全部授予。 TO root @＆＃39; 192.168.56.1＆＃39;通过＆＃39;密码识别＆＃39;; 查询正常，0行受影响，1警告（0.00秒）

的MySQL＆GT;冲洗特权; 查询正常，0行受影响（0.00秒）

的MySQL＆GT;出口

那是它，然后我去虚拟框内的客户ubuntu机器（通过这样做我的窗口与wifi互联网连接）并重新运行上面所说的sqoop命令，并导入我想要的表数据到群集。

我想分享成功运行脚本后得到的行 -

vm4learning@vm4learning:~/Installations/sqoop-1.4.4$ bin/sqoop import --connect jdbc:mysql://192.168.56.1/india --table india_most_populated_cities --target-dir /user/vm4learning/remotedir/ --username root --password password -m 1
no main manifest attribute, in /home/vm4learning/Installations/hbase-0.94.14/lib/coprocessor.jar
find: paths must precede expression: sqoop-test-1.4.4.jar
Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]
Warning: $HADOOP_HOME is deprecated.

17/12/06 17:00:55 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
17/12/06 17:00:55 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
17/12/06 17:00:55 INFO tool.CodeGenTool: Beginning code generation
17/12/06 17:00:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `india_most_populated_cities` AS t LIMIT 1
17/12/06 17:00:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `india_most_populated_cities` AS t LIMIT 1
17/12/06 17:00:58 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/vm4learning/Installations/hadoop-1.2.1
Note: /tmp/sqoop-vm4learning/compile/fc8e526de8f7a74171941455a22f573f/india_most_populated_cities.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/12/06 17:01:02 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-vm4learning/compile/fc8e526de8f7a74171941455a22f573f/india_most_populated_cities.jar
17/12/06 17:01:02 WARN manager.MySQLManager: It looks like you are importing from mysql.
17/12/06 17:01:02 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
17/12/06 17:01:02 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
17/12/06 17:01:02 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
17/12/06 17:01:02 INFO mapreduce.ImportJobBase: Beginning import of india_most_populated_cities
17/12/06 17:01:06 INFO mapred.JobClient: Running job: job_201712061003_0002
17/12/06 17:01:07 INFO mapred.JobClient:  map 0% reduce 0%
17/12/06 17:01:29 INFO mapred.JobClient:  map 100% reduce 0%
17/12/06 17:01:36 INFO mapred.JobClient: Job complete: job_201712061003_0002
17/12/06 17:01:37 INFO mapred.JobClient: Counters: 18
17/12/06 17:01:37 INFO mapred.JobClient:   Job Counters 
17/12/06 17:01:37 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=24525
17/12/06 17:01:37 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
17/12/06 17:01:37 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
17/12/06 17:01:37 INFO mapred.JobClient:     Launched map tasks=1
17/12/06 17:01:37 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
17/12/06 17:01:37 INFO mapred.JobClient:   File Output Format Counters 
17/12/06 17:01:37 INFO mapred.JobClient:     Bytes Written=10406
17/12/06 17:01:37 INFO mapred.JobClient:   FileSystemCounters
17/12/06 17:01:37 INFO mapred.JobClient:     HDFS_BYTES_READ=87
17/12/06 17:01:37 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=80423
17/12/06 17:01:37 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=10406
17/12/06 17:01:37 INFO mapred.JobClient:   File Input Format Counters 
17/12/06 17:01:37 INFO mapred.JobClient:     Bytes Read=0
17/12/06 17:01:37 INFO mapred.JobClient:   Map-Reduce Framework
17/12/06 17:01:37 INFO mapred.JobClient:     Map input records=271
17/12/06 17:01:37 INFO mapred.JobClient:     Physical memory (bytes) snapshot=81043456
17/12/06 17:01:37 INFO mapred.JobClient:     Spilled Records=0
17/12/06 17:01:37 INFO mapred.JobClient:     CPU time spent (ms)=2530
17/12/06 17:01:37 INFO mapred.JobClient:     Total committed heap usage (bytes)=49807360
17/12/06 17:01:37 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=973082624
17/12/06 17:01:37 INFO mapred.JobClient:     Map output records=271
17/12/06 17:01:37 INFO mapred.JobClient:     SPLIT_RAW_BYTES=87
17/12/06 17:01:37 INFO mapreduce.ImportJobBase: Transferred 10.1621 KB in 33.9572 seconds (306.4444 bytes/sec)
17/12/06 17:01:37 INFO mapreduce.ImportJobBase: Retrieved 271 records.

希望它会对你有所帮助，也能给我的帖子提供这个帖子的答案，谢谢。

Sqoop导入错误：用户访问被拒绝＆＃39; root＆＃39; localhost＆＃39;即使权利还可以

3 个答案: