Question

我已将数据库密码存储在Hadoop CredentialProvider中。

从终端导入Sqoop工作正常，从CredentialProvider成功获取密码。

sqoop import
-Dhadoop.security.credential.provider.path=jceks://hdfs/user/vijay/myPassword.jceks
--table myTable -m 1 --target-dir /user/vijay/output  --delete-target-dir --username vijay  --password-alias db2-dev-password

但是当我尝试设置为Sqoop作业时，它无法识别 - Dhadoop.security.credential.provider.path参数。

sqoop job --create my-sqoop-job -- import --table myTable -m 1 --target-dir /user/vijay/output  --delete-target-dir --username vijay  -Dhadoop.security.credential.provider.path=jceks://hdfs/user/vijay/myPassword.jceks --password-alias

以下是错误消息：

14/04/05 13:57:53 ERROR tool.BaseSqoopTool: Error parsing arguments for import:
14/04/05 13:57:53 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dhadoop.security.credential.provider.path=jceks://hdfs/user/vijay/myPassword.jceks
14/04/05 13:57:53 ERROR tool.BaseSqoopTool: Unrecognized argument: --password-alias
14/04/05 13:57:53 ERROR tool.BaseSqoopTool: Unrecognized argument: db2-dev-password

我在Sqoop User Guide中找不到使用Sqoop Job配置Hadoop凭证API的任何特殊说明。

如何解决此问题？

Answer 1

重新定位Sqoop参数可以解决问题。

sqoop job -Dhadoop.security.credential.provider.path=jceks://hdfs/user/vijay/myPassword.jceks --create my-sqoop-job -- import --table myTable -m 1 --target-dir /user/vijay/output  --delete-target-dir --username vijay   --password-alias myPasswordAlias

将Hadoop凭据放在Sqoop作业关键字之前。

Answer 2

您的Sqoop作业命令不正确，即--password-alias不完整请在Hadoop服务器中执行以下命令

hadoop credential list -provider jceks://hdfs/user/vijay/myPassword.jceks

在下面的Sqoop作业命令中添加输出

sqoop job --create my-sqoop-job -- import --table myTable -m 1 --target-dir /user/vijay/output  --delete-target-dir --username vijay  -Dhadoop.security.credential.provider.path=jceks://hdfs/user/vijay/myPassword.jceks --password-alias <<output of above command>>

Sqoop作业无法使用Hadoop Credential API

2 个答案: