Camel Route中的生产者流控制问题(持久性消息)

时间:2016-01-14 16:27:12

标签: java spring apache-camel activemq mq

我在查找正确的activemq配置集时遇到问题,以确保Apache Camel路由中的消息吞吐量一致。 当前配置使用以下技术:

  • Camel(2.15.2)
  • ActiveMQ(5.12.1)
  • Tomcat(7.0.56)

    以下是Camel for ActiveMQ中使用的bean配置集:

                          

    <bean id="jmsConnectionFactory" class="org.apache.activemq.spring.ActiveMQConnectionFactory">
        <property name="brokerURL" value="tcp://localhost:6616?jms.prefetchPolicy.queuePrefetch=100" />
         <property name="watchTopicAdvisories" value="false" />
         <property name="producerWindowSize" value="2300" />
    </bean>
    
    <bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory"
        init-method="start" destroy-method="stop">
        <property name="maxConnections" value="20" />
        <property name="connectionFactory" ref="jmsConnectionFactory" />
        <property name="idleTimeout" value="0"/>
    </bean>
    
    <bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
            <property name="connectionFactory" ref="pooledConnectionFactory"/> 
            <property name="transactionManager" ref="jmsTransactionManager"/> 
            <property name="transacted" value="true"/>
    

    - &gt;                  

    <bean id="jmsTransactionManager" class="org.springframework.jms.connection.JmsTransactionManager">
            <property name="connectionFactory" ref="jmsConnectionFactory" />
    </bean>
    
    <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
        <property name="configuration" ref="jmsConfig" />
    </bean>
    

以下是activemq.xml文件中找到的代理特定配置:

<broker xmlns="http://activemq.apache.org/schema/core"
        brokerName="localhost" dataDirectory="./activemq/data/" advisorySupport="false">
        <destinationPolicy>
            <policyMap>
                <policyEntries>
                    <policyEntry queue="PICKAXE.L5.PROC.>" producerFlowControl="true" storeUsageHighWaterMark="50" />
                    <policyEntry queue="PICKAXE.L5.COL.>" producerFlowControl="true" storeUsageHighWaterMark="95" />
                </policyEntries>
            </policyMap>
        </destinationPolicy>

        <managementContext>
            <managementContext createConnector="true" />
        </managementContext>

        <persistenceAdapter>
            <kahaDB directory="./activemq/kahadb/" />
        </persistenceAdapter>

        <systemUsage>
            <systemUsage sendFailIfNoSpaceAfterTimeout="3000000">
                <memoryUsage>
                    <memoryUsage limit="750 mb" />
                </memoryUsage>
                <storeUsage>
                    <storeUsage limit="2 gb" />
                </storeUsage>
                <tempUsage>
                    <tempUsage limit="500 mb" />
                </tempUsage>
            </systemUsage>
        </systemUsage>

        <transportConnectors>

            <transportConnector name="openwire"
                uri="tcp://0.0.0.0:6616?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600" />
            <transportConnector name="amqp"
                uri="amqp://0.0.0.0:5672?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600" />
        </transportConnectors>
        <shutdownHooks>
            <bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook" />
        </shutdownHooks>
    </broker>

我正在运行下面的Camel路线。队列A接收大量消息(1000 / s),因此它会很快开始填充,因为这些消息的最终消费者无法跟上。当消息量最终达到持久存储生成器流量控制规则的50%时,防止进一步的消息被放置在队列A上。但是,当我通过JMX检查队列深度时,队列A和B都不会像消费者一样改变也被封锁了。

    from(activemq:queue:PICKAXE.L5.PROC.A)
        .to(activemq:queue:PICKAXE.L5.COL.B);

    from(activemq:queue:PICKAXE.L5.COL.B)
        .autoStartup(!localFlag)
        .to(customEndpoint)
        .routeId(collectionRouteId);

大约一个星期我尝试了jms / activemq配置的各种排列,没有运气,所以我会感激任何想法。我追求的期望行为是让此流程中的消息使用者继续从持久存储中删除消息,这将允许消息继续流畅。

1 个答案:

答案 0 :(得分:0)

问题是由于在上述配置中设置为 3000000 sendFailIfNoSpaceAfterTimeout 过大造成的。这导致代理在确认由于持久存储已满而导致send()命令失败之前等待。

上述配置已替换为以下内容:

<systemUsage sendFailIfNoSpaceAfterTimeout="300">

这确保了(由于消息是持久性的并且队列被集成到Camel路由中)当持久性存储器已满时导致失败时,每0.3秒重试一次send()操作。