Question

我最近一直在试验Flume。我目前正在使用JMS源（http://flume.apache.org/FlumeUserGuide.html#jms-source）

但是，当我尝试运行flume代理时，它会成功创建通道和接收器，但在创建Source时失败。我使用的是TIBCO EMS，我需要使用EMS用户名/密码组合进行身份验证。

文档要求我定义一个passwordFile，其中包含目标/提供者的密码。是否有正确的方法来创建此文件（我目前将密码存储在文本文件中）。

这是错误堆栈（我非常确定用户/密码是否正确）

13/11/17 11:49:08 INFO source.DefaultSourceFactory: Creating instance of source WeatherData, type jms
13/11/17 11:49:08 ERROR node.AbstractConfigurationProvider: Source WeatherData has been removed due to an error during configuration
org.apache.flume.FlumeException: Could not lookup ConnectionFactory
        at org.apache.flume.source.jms.JMSSource.doConfigure(JMSSource.java:222)
        at org.apache.flume.source.BasicSourceSemantics.configure(BasicSourceSemantics.java:65)
        at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
        at org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:331)
        at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
        at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: javax.naming.AuthenticationException: Not permitted: invalid name or password [Root exception is javax.jms.JMSSecurityException: invalid name or password]
        at com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:668)
        at com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:489)
        at javax.naming.InitialContext.lookup(InitialContext.java:392)
        at org.apache.flume.source.jms.JMSSource.doConfigure(JMSSource.java:219)
        ... 14 more
Caused by: javax.jms.JMSSecurityException: invalid name or password
        at com.tibco.tibjms.Tibjmsx.buildException(Tibjmsx.java:543)
        at com.tibco.tibjms.TibjmsConnection._create(TibjmsConnection.java:1044)
        at com.tibco.tibjms.TibjmsConnection.<init>(TibjmsConnection.java:2707)
        at com.tibco.tibjms.TibjmsQueueConnection.<init>(TibjmsQueueConnection.java:36)
        at com.tibco.tibjms.TibjmsxCFImpl._createImpl(TibjmsxCFImpl.java:186)
        at com.tibco.tibjms.TibjmsxCFImpl._createConnection(TibjmsxCFImpl.java:239)
        at com.tibco.tibjms.TibjmsQueueConnectionFactory.createQueueConnection(TibjmsQueueConnectionFactory.java:87)
        at com.tibco.tibjms.naming.TibjmsContext$Messenger.request(TibjmsContext.java:325)
        at com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:655)
        ... 17 more

Answer 1

关于passwordFile不确定，这似乎是Flume特有的。 TIBCO EMS在2个级别进行身份验证：

查找connectionFactory
查找目的地

一些像“春天”这样的“预建”配置（并且可能是水槽）与此斗争并需要一些返工。

检查下面的代码，您可以在两个级别上看到用户/ pw被应用来自TIBCO EMS的Queue queue.sample：



/**
 * @param args
 */
public static void main(String[] args) {

    String userName = "user";
    String password = "password";
    try {
        // Obtain a JNDI connection
        Properties env = new Properties();
        env.put(Context.INITIAL_CONTEXT_FACTORY,
                "com.tibco.tibjms.naming.TibjmsInitialContextFactory");
        env.put(Context.PROVIDER_URL, "tibjmsnaming://localhost:7222");
        env.put(Context.SECURITY_PRINCIPAL, userName);
        env.put(Context.SECURITY_CREDENTIALS, password);
        // ... specify the JNDI properties specific to the vendor
        InitialContext jndi = new InitialContext(env);

        ConnectionFactory factory = (ConnectionFactory) jndi
                .lookup("ConnectionFactory");

        try {
            Connection connection = factory.createConnection(userName,
                    password);
            Session session = connection.createSession();

            Queue sampleTopic = (Queue) jndi.lookup("queue.sample");

        } catch (JMSException je) {
            je.printStackTrace();

        }

    } catch (NamingException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

}

/** * @param args */ public static void main(String[] args) { String userName = "user"; String password = "password"; try { // Obtain a JNDI connection Properties env = new Properties(); env.put(Context.INITIAL_CONTEXT_FACTORY, "com.tibco.tibjms.naming.TibjmsInitialContextFactory"); env.put(Context.PROVIDER_URL, "tibjmsnaming://localhost:7222"); env.put(Context.SECURITY_PRINCIPAL, userName); env.put(Context.SECURITY_CREDENTIALS, password); // ... specify the JNDI properties specific to the vendor InitialContext jndi = new InitialContext(env); ConnectionFactory factory = (ConnectionFactory) jndi .lookup("ConnectionFactory"); try { Connection connection = factory.createConnection(userName, password); Session session = connection.createSession(); Queue sampleTopic = (Queue) jndi.lookup("queue.sample"); } catch (JMSException je) { je.printStackTrace(); } } catch (NamingException e) { // TODO Auto-generated catch block e.printStackTrace(); } }

Answer 2

如果密码文件包含特殊字符（如重音符号），请确保密码文件以UTF-8编码。 Flume使用这个字符集读取文件。

Answer 3

Flume源中存在一个缺陷，它无法将凭据正确传递给JNDI服务器。 FLUME-2095解决了这个问题，并提出了补丁。他们指定用户名/密码的方式可能没什么问题，因为如果没有这个修复程序，它将无法工作。

我所知道的唯一解决方法是在JMS级别关闭访问控制，这样就完全不需要凭据，但会使您的队列/主题保持开放状态。

Flume - 如何为JMS Source（Tibco）指定用户名/密码

3 个答案: