TD-AGENT + WebHDFS + kerberos =错误“ webhdfs检查请求失败”

时间:2019-01-10 08:54:43

标签: kerberos fluentd webhdfs td-agent

有一个错误:尝试使用fluentd(td-agent)通过WebHDFS插件将日志写入HDFS,但不能这样做。因为它们之间存在愤怒的警卫-Kerberos:我无法通过密钥表进行身份验证。

2018-11-28 20:22:19 +0300 [error]: #0 /usr/sbin/td-agent:7:in `<main>'
2018-11-28 20:22:19 +0300 [error]: #0 unexpected error error_class=RuntimeError error="webhdfs is not available now."
  2018-11-28 20:22:19 +0300 [error]: #0 suppressed same stacktrace
2018-11-28 20:22:19 +0300 [info]: Worker 0 finished unexpectedly with status 1
2018-11-28 20:22:20 +0300 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
2018-11-28 20:22:20 +0300 [info]: gem 'fluent-plugin-elasticsearch' version '2.12.1'
2018-11-28 20:22:20 +0300 [info]: gem 'fluent-plugin-kafka' version '0.6.1'
2018-11-28 20:22:20 +0300 [info]: gem 'fluent-plugin-mongo' version '0.8.1'
2018-11-28 20:22:20 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.5.6'
2018-11-28 20:22:20 +0300 [info]: gem 'fluent-plugin-s3' version '0.8.5'
2018-11-28 20:22:20 +0300 [info]: gem 'fluent-plugin-scribe' version '0.10.14'
2018-11-28 20:22:20 +0300 [info]: gem 'fluent-plugin-td' version '0.10.29'
2018-11-28 20:22:20 +0300 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.3'
2018-11-28 20:22:20 +0300 [info]: gem 'fluent-plugin-webhdfs' version '1.2.3'
2018-11-28 20:22:20 +0300 [info]: gem 'fluent-plugin-webhdfs' version '0.7.1'
2018-11-28 20:22:20 +0300 [info]: gem 'fluentd' version '1.3.0'
2018-11-28 20:22:20 +0300 [info]: gem 'fluentd' version '0.12.40'
2018-11-28 20:22:20 +0300 [info]: adding match pattern="hdfs.*.*" type="webhdfs"
2018-11-28 20:22:20 +0300 [warn]: #0 'flush_interval' is ignored because default 'flush_mode' is not 'interval': 'lazy'
2018-11-28 20:22:20 +0300 [info]: adding source type="http"
2018-11-28 20:22:20 +0300 [info]: adding source type="debug_agent"
2018-11-28 20:22:20 +0300 [info]: #0 starting fluentd worker pid=5286 ppid=59682 worker=0
2018-11-28 20:22:20 +0300 [warn]: #0 webhdfs check request failed. (namenode: hdp85:50070, error: gss_init_sec_context did not return GSS_S_COMPLETE: Unspecified GSS failure.  Minor code may provide more information
Ticket expired
)

因此,在这种情况下,我可以手动登录:

sudo -u td-agent bash
export KRB5_CONFIG=/etc/krb5.conf
kinit -kt /var/lib/td-agent/td-agent.keytab td-agent
  1. 将用户更改为流利用户
  2. 设置$ path变量
  3. Kinit

有效。但是,每当我需要使用流利的(测试)时,我应该手动进行操作。通过配置文件(kerberos_keytab "/var/lib/td-agent/td-agent.keytab")进行自动切换不起作用。那么,如何在生产中使用它?

0 个答案:

没有答案