我正在尝试将Websphere（IBM）MQ与flume集成。我有几个xml文件将来自MQ＆amp;我希望得到这些消息。在Spark中进行进一步处理。
我正在安装我的Hadoop的AWS EC2实例上进行此集成。 以下是我进行整合的步骤。
./ crtmqm -q -d MY.DEFAULT.BTS.QUEUE -u SYSTEM.DEAD.LETTER.QUEUE CSBTS.QUEUE.MANAGER
启动队列管理器：
./ strmqm CSBTS.QUEUE.MANAGER
./runmqsc CSBTS.QUEUE.MANAGER
定义服务器频道
DEFINE CHANNEL（CHAN2）CHLTYPE（SVRCONN）TRPTYPE（TCP）DESCR（'服务器连接到Client_2'）
DEFINE CHANNEL（CHAN2）CHLTYPE（CLNTCONN）TRPTYPE（TCP）CONNAME（10.0.0.238）QMNAME（CSBTS.QUEUE.MANAGER）DESCR（'客户端连接到Server_2'）
ALTER QMGR CHLAUTH（DISABLED）
定义本地队列
DEFINE QLOCAL（CSBTS.DEAL）
添加消息
./ amqsput CSBTS.DEAL CSBTS.QUEUE.MANAGER
创建连接工厂
DEF CF（CF4）QMGR（CSBTS.QUEUE.MANAGER）TRANSPORT（客户）HOSTNAME（10.0.0.238）CHANNEL（CHAN2）
启动侦听器
DEFINE LISTENER（CSBTS_LIST）TRPTYPE（TCP）PORT（1414）CONTROL（QMGR）
DISPLAY LISTENER（CSBTS_LIST）
DISPLAY LSSTATUS（CSBTS_LIST）
（RESTART qm）
跑水槽
/ usr / bin / flume-ng agent -c / etc / flume-ng / conf -f /etc/flume-ng/conf/flume.conf -n tier1
Flume通过linux用户“hduser”执行。
以下是我们获得的身份验证问题。
20 Jul 2016 09:17:33,236 ERROR [PollableSourceRunner-JMSSource-source1] (org.apache.flume.source.PollableSourceRunner$PollingRunner.run:156) - Unhandled exception, logging and sleeping for 5000ms
org.apache.flume.FlumeException: Source had error configuring or starting
at org.apache.flume.source.AbstractPollableSource.process(AbstractPollableSource.java:48)
at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:139)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.flume.FlumeException: Could not create connection to broker
at org.apache.flume.source.jms.JMSMessageConsumer.<init>(JMSMessageConsumer.java:76)
at org.apache.flume.source.jms.JMSMessageConsumerFactory.create(JMSMessageConsumerFactory.java:32)
at org.apache.flume.source.jms.JMSSource.createConsumer(JMSSource.java:317)
at org.apache.flume.source.jms.JMSSource.doStart(JMSSource.java:299)
at org.apache.flume.source.BasicSourceSemantics.start(BasicSourceSemantics.java:83)
at org.apache.flume.source.PollableSourceRunner.start(PollableSourceRunner.java:74)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Caused by: com.ibm.msg.client.jms.DetailedJMSSecurityException: JMSWMQ2013: The security authentication was not valid that was supplied for QueueManager 'CSBTS.QUEUE.MANAGER' with connection mode 'Client' and host name '10.0.0.238(1414)'.
Please check if the supplied username and password are correct on the QueueManager to which you are connecting.
at com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:531)
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:215)
at com.ibm.msg.client.wmq.internal.WMQConnection.<init>(WMQConnection.java:413)
at com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createV7ProviderConnection(WMQConnectionFactory.java:8475)
at com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createProviderConnection(WMQConnectionFactory.java:7814)
at com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl._createConnection(JmsConnectionFactoryImpl.java:299)
at com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl.createConnection(JmsConnectionFactoryImpl.java:236)
at com.ibm.mq.jms.MQConnectionFactory.createCommonConnection(MQConnectionFactory.java:6024)
at com.ibm.mq.jms.MQConnectionFactory.createConnection(MQConnectionFactory.java:6073)
at org.apache.flume.source.jms.JMSMessageConsumer.<init>(JMSMessageConsumer.java:69)
... 13 more
Caused by: com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2035' ('MQRC_NOT_AUTHORIZED').
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:203)
... 21 more
过去3周我坚持这个，如果有人遇到这个问题，请帮助我们。
来自AMQERR01.LOG的错误日志
----- cmqxrsrv.c : 2321 -------------------------------------------------------
07/21/2016 10:08:17 AM - Process(16169.50) User(mqm) Program(amqzlaa0)
Host(ip-10-0-0-238) Installation(Installation1)
VRMF(8.0.0.4) QMgr(CSBTS.QUEUE.MANAGER)
AMQ5540: Application 'flume.node.Application' did not supply a user ID and
password
EXPLANATION:
The queue manager is configured to require a user ID and password, but none was
supplied.
ACTION:
Ensure that the application provides a valid user ID and password, or change
the queue manager configuration to OPTIONAL to allow applications to connect
which have not supplied a user ID and password.
----- amqzfuca.c : 4287 -------------------------------------------------------
07/21/2016 10:08:17 AM - Process(16169.50) User(mqm) Program(amqzlaa0)
Host(ip-10-0-0-238) Installation(Installation1)
VRMF(8.0.0.4) QMgr(CSBTS.QUEUE.MANAGER)
AMQ5541: The failed authentication check was caused by the queue manager
CONNAUTH CHCKCLNT(REQDADM) configuration.
EXPLANATION:
The user ID 'mqm' and its password were checked because the user ID is
privileged and the queue manager connection authority (CONNAUTH) configuration
refers to an authentication information (AUTHINFO) object named
'SYSTEM.DEFAULT.AUTHINFO.IDPWOS' with CHCKCLNT(REQDADM).
This message accompanies a previous error to clarify the reason for the user ID
and password check.
ACTION:
Refer to the previous error for more information.
Ensure that a password is specified by the client application and that the
password is correct for the user ID. The authentication configuration of the
queue manager connection determines the user ID repository. For example, the
local operating system user database or an LDAP server.
To avoid the authentication check, you can either use an unprivileged user ID
or amend the authentication configuration of the queue manager. You can amend
the CHCKCLNT attribute in the CHLAUTH record, but you should generally not
allow unauthenticated remote access.
-------------------------------------------------------------------------------
07/21/2016 10:08:18 AM - Process(18280.13) User(mqm) Program(amqrmppa)
Host(ip-10-0-0-238) Installation(Installation1)
VRMF(8.0.0.4) QMgr(CSBTS.QUEUE.MANAGER)
AMQ9557: Queue Manager User ID initialization failed for 'mqm'.
EXPLANATION:
The call to initialize the User ID 'mqm' failed with CompCode 2 and Reason
2035.
ACTION:
Correct the error and try again.
----- cmqxrsrv.c : 2321 -------------------------------------------------------
07/21/2016 10:10:58 AM - Process(16169.51) User(mqm) Program(amqzlaa0)
Host(ip-10-0-0-238) Installation(Installation1)
VRMF(8.0.0.4) QMgr(CSBTS.QUEUE.MANAGER)
AMQ8077: Entity 'hduser' has insufficient authority to access object
'CSBTS.QUEUE.MANAGER'.
EXPLANATION:
The specified entity is not authorized to access the required object. The
following requested permissions are unauthorized: connect
ACTION:
Ensure that the correct level of authority has been set for this entity against
the required object, or ensure that the entity is a member of a privileged
group.
----- amqzfubx.c : 687 --------------------------------------------------------
07/21/2016 10:10:58 AM - Process(18280.14) User(mqm) Program(amqrmppa)
Host(ip-10-0-0-238) Installation(Installation1)
VRMF(8.0.0.4) QMgr(CSBTS.QUEUE.MANAGER)
AMQ9557: Queue Manager User ID initialization failed for 'hduser'.
EXPLANATION:
The call to initialize the User ID 'hduser' failed with CompCode 2 and Reason
2035.
ACTION:
Correct the error and try again.
----- cmqxrsrv.c : 2321 -------------------------------------------------------
答案 0 :(得分：0)
感谢@MoragHughson，根据您的建议，寻找AMQERR01.LOG以找出更多问题。它有助于克服我使用Flume Interating MQ的问题。
我所做的是将队列管理器配置更改为OPTIONAL以允许应用程序连接但未提供用户ID和密码。在我的情况下，我没有在我的flume.conf文件中传递密码。
我点了这个链接 - http://www-01.ibm.com/support/docview.wss?uid=swg21680930 并使用选项B.
以下是我用来进行身份验证的命令 -
ALTER AUTHINFO(SYSTEM.DEFAULT.AUTHINFO.IDPWOS) AUTHTYPE(IDPWOS) CHCKCLNT(OPTIONAL)
REFRESH SECURITY TYPE(CONNAUTH)
我希望这对尝试将MQ与FLume / Kakfa集成的任何人都有帮助。