Question

在我的春季批处理中，我使用的是Oracle DB和IBM WMQ，并实现了Atomikos事务管理和Spring Partitioning。

要求是，需要从数据库中读取200万条记录，并在处理后，将相同的消息（1条消息大小= 2kb）发布到WMQ中，并在数据库中更新200万条记录。

以下是我的代码配置。队列配置： `

<bean id="atomikosJmsConnectionFactory" class="com.atomikos.jms.AtomikosConnectionFactoryBean" init-method="init" destroy-method="close">
        <property name="uniqueResourceName" value="WMQ_MQSeries_XA_RMI" />
        <property name="xaConnectionFactory">
            <bean class="com.ibm.mq.jms.MQXAConnectionFactory">
                <property name="connectionNameList" value="127.0.0.1(1414)" />
                <property name="queueManager" value="LWMQ" />
                <property name="channel" value="LCHNL" />
                <property name="transportType">
                  <util:constant static-field="com.ibm.msg.client.wmq.WMQConstants.WMQ_CM_CLIENT" />
                </property>
                <property name="clientReconnectOptions">
                  <util:constant static-field="com.ibm.msg.client.wmq.WMQConstants.WMQ_CLIENT_RECONNECT" />
                </property>
            </bean>
        </property>
        <property name="ignoreSessionTransactedFlag" value="${mq.ignoreSessionTransactedFlag:false}"></property>
        <property name="borrowConnectionTimeout" value="300"></property>
        <property name="maxPoolSize" value="50"></property>
    </bean>

     <bean id="jmsTemplate"  class="org.springframework.jms.core.JmsTemplate">
        <property name="connectionFactory" ref="atomikosJmsConnectionFactory" />
        <property name="sessionTransacted" value="true"/>
        <property name="sessionAcknowledgeMode" value="0" />
        <property name="receiveTimeout" value="2000" />
    </bean>

`

Spring作业配置：

<batch:step id="masterStep">
      <batch:partition step="slaveStep" partitioner="customPartitioner">
        <batch:handler task-executor="taskExecutor" />
      </batch:partition>
    </batch:step>
    <batch:listeners>
      <batch:listener ref="loadJobListener" />
    </batch:listeners>
  </batch:job>

  <batch:step id="slaveStep">
    <batch:tasklet transaction-manager="atomikosTransactionManager">
      <batch:chunk chunk-completion-policy="chunkCompletionPolicy" />
    </batch:tasklet>
  </batch:step>

<bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor" >
    <property name="threadNamePrefix" value="L-Thread-"></property>
    <property name="concurrencyLimit" value="20"></property>
  </bean>

 <bean id="chunkTimeout"
    class="org.springframework.batch.repeat.policy.TimeoutTerminationPolicy">
    <constructor-arg value="36000" />
  </bean>

  <bean id="commitCount"
    class="org.springframework.batch.repeat.policy.SimpleCompletionPolicy">
    <property name="chunkSize" value="100" />
  </bean>

<bean id="chunkCompletionPolicy"
    class="org.springframework.batch.repeat.policy.CompositeCompletionPolicy">
    <property name="policies">
      <list>
        <ref bean="commitCount" />
        <ref bean="chunkTimeout" />
      </list>
    </property>
  </bean>

这是我的linux服务器配置：型号：Intel（R）Xeon（R）CPU v4 @ 2.60GHz 架构：x86_64 没有CPU：16 RAM：100GB

目前，该工作需要120分钟才能完成任务（包括200万封邮件发布到MQ并更新到数据库中）

我的观察：大多数情况下，线程处于等待状态以进行MQ连接。以下是threaddump的Visual-VM截图。

"L-Thread-4" #101 prio=5 os_prio=0 tid=0x000000002349e000 nid=0xcabc waiting on condition [0x00000000324cd000]
   java.lang.Thread.State: TIMED_WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x0000000771afc540> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at com.ibm.ws.xs.xio.actor.impl.FutureImpl.await(FutureImpl.java:268)
    at com.ibm.ws.xs.xio.actor.impl.FutureImpl.get(FutureImpl.java:316)
    at com.ibm.ws.xsspi.xio.dispatch.MessageDispatcher.askAndGet(MessageDispatcher.java:373)
    at com.ibm.ws.xs.xio.actor.impl.ActorRefImpl.askAndGet(ActorRefImpl.java:106)
    at com.ibm.ws.objectgrid.client.XIOClientCoreMessageHandler.sendMessageInternal(XIOClientCoreMessageHandler.java:372)
    at com.ibm.ws.objectgrid.client.XIOClientCoreMessageHandler.sendMessage(XIOClientCoreMessageHandler.java:142)
    at com.ibm.ws.objectgrid.client.CommonClientCoreMessageHandler.sendGetRequest(CommonClientCoreMessageHandler.java:229)
    at com.ibm.ws.objectgrid.client.RemoteTransactionCallbackImpl.mapGet(RemoteTransactionCallbackImpl.java:1953)
    at com.ibm.ws.objectgrid.client.RemoteCacheLoader.get(RemoteCacheLoader.java:497)
    at com.ibm.ws.objectgrid.map.BaseMap.getRaw(BaseMap.java:4846)
    at com.ibm.ws.objectgrid.DiffMap.lookForKey(DiffMap.java:1391)
    at com.ibm.ws.objectgrid.DiffMap.get(DiffMap.java:981)
    at com.ibm.ws.objectgrid.ObjectMapImpl.get(ObjectMapImpl.java:451)
    at com.ibm.ws.objectgrid.ObjectMapImpl.get(ObjectMapImpl.java:411)
    at com.ibm.websphere.objectgrid.spring.ObjectGridCache.get(ObjectGridCache.java:290)

如何解决此问题以加快我的流程..？

我需要在1小时内完成这个工作流程。

Answer 1

您应该使用preventDefault()来避免为每次发送打开新连接。

JMS队列连接的性能

1 个答案: