Question

我正在开发Java应用程序，该应用程序正在将结果数据保存到HDFS。 java应用程序应该在我的Windows机器上运行。

我们使用Kerberos身份验证，并将密钥表文件放在NAS驱动器中。我们将Hadoop配置文件保存在同一个NAS驱动器中。

我的问题是当我从NAS驱动器加载Hadoop配置文件时，它给我一些验证错误，但是如果我从本地文件系统加载配置文件，我的应用程序运行正常（我还在C中保存了配置文件）：\ Hadoop的）

以下是我的工作代码段。 （NAS中的keytab文件，本地文件系统中的Hadoop配置文件）

static String KeyTabPath = "\\\\path\\2\\keytabfile\\name.keytab"
Configuration config = new Configuration();
        config.set("fs.defaultFS", "hdfs://xxx.xx.xx.com:8020");
        config.addResource(new Path("C:\\Hadoop\\core-site.xml"));
        config.addResource(new Path("C:\\Hadoop\\hdfs-site.xml"));
        config.addResource(new Path("C:\\Hadoop\\mapred-site.xml"));
        config.addResource(new Path("C:\\Hadoop\\yarn-site.xml"));
        config.set("fs.hdfs.impl", org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());
        config.set("fs.file.impl",org.apache.hadoop.fs.LocalFileSystem.class.getName());
        // Kerberos Authentication
        config.set("hadoop.security.authentication", "Kerberos");
        UserGroupInformation.setConfiguration(config);
        UserGroupInformation.loginUserFromKeytab("name@xx.xx.COM",KeyTabPath);

我也尝试从NAS驱动器加载配置文件，但是获得了kerberos身份验证错误。下面是抛出错误的代码片段（NAS中的Keytab文件和NAS中的Hadoop配置文件）

static String KeyTabPath = "\\\\path\\2\\keytabfile\\name.keytab"
Configuration config = new Configuration();
        config.set("fs.defaultFS", "hdfs://xxx.xx.xx.com:8020");
        config.addResource(new Path("\\\\NASDrive\\core-site.xml"));
        config.addResource(new Path("\\\\NASDrive\\hdfs-site.xml"));
        config.addResource(new Path("\\\\NASDrive\\mapred-site.xml"));
        config.addResource(new Path("\\\\NASDrive\\yarn-site.xml"));
        config.set("fs.hdfs.impl", org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());
        config.set("fs.file.impl",org.apache.hadoop.fs.LocalFileSystem.class.getName());
        // Kerberos Authentication
        config.set("hadoop.security.authentication", "Kerberos");
        UserGroupInformation.setConfiguration(config);
        UserGroupInformation.loginUserFromKeytab("name@xx.xx.COM",KeyTabPath);

以下是错误讯息

java.io.IOException: Login failure for name@XX.XX.COM from keytab \\NASdrive\name.keytab: javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: Illegal principal name name@XX.XX.COM: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to name@XX.XX.COM
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:962)
    at Appname.ldapLookupLoop(Appname.java:111)
    at Appname.main(Appname.java:70)
Caused by: javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: Illegal principal name name@XX.XX.COM: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to name@XX.XX.COM
    at org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:199)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at javax.security.auth.login.LoginContext.invoke(Unknown Source)
    at javax.security.auth.login.LoginContext.access$000(Unknown Source)
    at javax.security.auth.login.LoginContext$4.run(Unknown Source)
    at javax.security.auth.login.LoginContext$4.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.login.LoginContext.invokePriv(Unknown Source)
    at javax.security.auth.login.LoginContext.login(Unknown Source)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:953)
    ... 2 more
Caused by: java.lang.IllegalArgumentException: Illegal principal name name@XX.XX.COM: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to name@XX.XX.COM
    at org.apache.hadoop.security.User.<init>(User.java:51)
    at org.apache.hadoop.security.User.<init>(User.java:43)
    at org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:197)
    ... 14 more
Caused by: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to name@XX.XX.COM
    at org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:389)
    at org.apache.hadoop.security.User.<init>(User.java:48)
    ... 16 more
Jul 06, 2016 4:29:14 PM com.XX.it.logging.JdkMapper info
INFO:  IO Exception occured: java.io.IOException: Login failure for name@XX.XX.COM from keytab \\NASdrive\name.keytab: javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: Illegal principal name name@XX.XX.COM: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to name@XX.XX.COM

所以问题似乎是加载配置文件。我的应用程序从NAS驱动器读取keytab文件，但不是Hadoop配置文件。可能是什么问题。我检查了所有NAS驱动器权限和文件权限。 Everthing很好。我不知道问题出在哪里。请任何人帮我找出问题。

Answer 1

您缺少auth_to_local Kerberos主体名称转换的“ DEFAULT”规则。

org.apache.hadoop.security.authentication.util.KerberosName $ NoMatchingRule：     没有适用于
的规则

在此处查看示例-

https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SecureMode.html#Mapping_from_Kerberos_principals_to_OS_user_accounts

因此，基本上只需在hadoop.security.auth_to_local的{{1}}的末尾添加单词“ DEFAULT”。

也请检查Kerberos documentation中的core-site.xml。

PS。这是where this exception happens in Hadoop codebase，以备您有兴趣深入研究此主题时使用。

Kerberos身份验证错误 - 从SharedPath加载Hadoop配置文件时

1 个答案: