大消息丢失,无法转移

时间:2019-07-26 07:27:14

标签: activemq-artemis

我在集群中设置了以下队列:

  • server1在61616上运行
  • server2在61617上运行

我使用两个不同的可用主服务器启动两个使用者

java -jar AmqJmsConsume.jar -duration 5 -queue IT-InputQueue -stats -log
/tmp/artemis/4 -verify -commitdelay 300 -url 'tcp://localhost:61616'

java -jar AmqJmsConsume.jar -duration 5 -queue IT-InputQueue -stats -log
/tmp/artemis/4 -verify -commitdelay 300 -url 'tcp://localhost:61617' 

并将生产者运行到server1,如下所示

java -jar AmqJmsProducer.jar -topic RFTopic -stats -log /tmp/artemis/4 -id
-count 500 -n 500 -ttl 3600000 -url 'tcp://localhost:61616' -outliers 100
-outliersize 500k

两个使用者都收到很少的消息,但是在以下异常服务器1清除之后 队列中有将近50%的邮件,但其余邮件被卡住了 internal.sf.cluster队列,并且未被任何使用者接收。

java.lang.IllegalStateException: no queueIDs defined
        at org.apache.activemq.artemis.core.server.cluster.impl.ClusterConnectionBridge.beforeForward(ClusterConnectionBridge.java:180) [artemis-server-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.core.server.cluster.impl.BridgeImpl.handle(BridgeImpl.java:609) [artemis-server-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.handle(QueueImpl.java:2983) [artemis-server-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:2334) [artemis-server-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2000(QueueImpl.java:107) [artemis-server-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:3209) [artemis-server-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66) [artemis-commons-2.6.3.jar:2.6.3]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_191]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_191]
        at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.6.3.jar:2.6.3]
2019-01-08 15:15:33,668 WARN [org.apache.activemq.artemis.core.server.impl.QueueImpl] null:
java.util.NoSuchElementException
        at org.apache.activemq.artemis.utils.collections.PriorityLinkedListImpl$PriorityLinkedListIterator.repeat(PriorityLinkedListImpl.java:172) [artemis-commons-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:2353) [artemis-server-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2000(QueueImpl.java:107) [artemis-server-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:3209) [artemis-server-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.6.3.jar:2.6.3]
        at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66) [artemis-commons-2.6.3.jar:2.6.3]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_191]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_191]
        at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.6.3.jar:2.6.3] 

server1-master经纪人

<?xml version='1.0'?>
<configuration xmlns="urn:activemq"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:xi="http://www.w3.org/2001/XInclude"
    xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">

    <core xmlns="urn:activemq:core"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:activemq:core ">

        <persistence-enabled>true</persistence-enabled>
        <thread-pool-max-size>400</thread-pool-max-size>
        <journal-type>NIO</journal-type>

        <paging-directory>${data.dir}/paging</paging-directory>

        <bindings-directory>${data.dir}/bindings</bindings-directory>

        <journal-directory>${data.dir}/journal</journal-directory>

        <large-messages-directory>${data.dir}/large-messages
        </large-messages-directory>

        <journal-datasync>true</journal-datasync>

        <journal-min-files>20</journal-min-files>

        <journal-pool-files>20</journal-pool-files>

        <journal-file-size>10M</journal-file-size>

        <journal-compact-min-files>30</journal-compact-min-files>

        <journal-buffer-timeout>23480000</journal-buffer-timeout>


        <!-- When using ASYNCIO, this will determine the writing queue depth for 
            libaio. -->
        <journal-max-io>1</journal-max-io>
        <!-- You can verify the network health of a particular NIC by specifying 
            the <network-check-NIC> element. <network-check-NIC>theNicName</network-check-NIC> -->

        <!-- Use this to use an HTTP server to validate the network <network-check-URL-list>http://www.apache.org</network-check-URL-list> -->

        <!-- <network-check-period>10000</network-check-period> -->
        <!-- <network-check-timeout>1000</network-check-timeout> -->

        <!-- this is a comma separated list, no spaces, just DNS or IPs it should 
            accept IPV6 Warning: Make sure you understand your network topology as this 
            is meant to validate if your network is valid. Using IPs that could eventually 
            disappear or be partially visible may defeat the purpose. You can use a list 
            of multiple IPs, and if any successful ping will make the server OK to continue 
            running -->
        <!-- <network-check-list>10.0.0.1</network-check-list> -->

        <!-- use this to customize the ping used for ipv4 addresses -->
        <!-- <network-check-ping-command>ping -c 1 -t %d %s</network-check-ping-command> -->

        <!-- use this to customize the ping used for ipv6 addresses -->
        <!-- <network-check-ping6-command>ping6 -c 1 %2$s</network-check-ping6-command> -->




        <!-- how often we are looking for how many bytes are being used on the 
            disk in ms -->
        <disk-scan-period>5000</disk-scan-period>

        <!-- once the disk hits this limit the system will block, or close the 
            connection in certain protocols that won't support flow control. -->
        <max-disk-usage>90</max-disk-usage>

        <!-- should the broker detect dead locks and other issues -->
        <critical-analyzer>true</critical-analyzer>

        <critical-analyzer-timeout>120000</critical-analyzer-timeout>

        <critical-analyzer-check-period>60000</critical-analyzer-check-period>

        <critical-analyzer-policy>LOG</critical-analyzer-policy>

        <transaction-timeout>1800000</transaction-timeout>
        <!-- the system will enter into page mode once you hit this limit. This 
            is an estimate in bytes of how much the messages are using in memory The 
            system will use half of the available memory (-Xmx) by default for the global-max-size. 
            You may specify a different value here if you need to customize it to your 
            needs. <global-max-size>100Mb</global-max-size> -->
        <connectors>
            <connector name="netty-connector">tcp://localhost:61616</connector>
        </connectors>

        <!-- Acceptors -->
        <acceptors>
            <acceptor name="netty-acceptor">tcp://localhost:61616?anycastPrefix=jms.queue.;multicastPrefix=jms.topic.</acceptor>
        </acceptors>


        <ha-policy>
            <shared-store>
                <master>
                    <failover-on-shutdown>true</failover-on-shutdown>
                </master>
            </shared-store>
        </ha-policy>

        <broadcast-groups>
            <broadcast-group name="my-broadcast-group">
                <broadcast-period>5000</broadcast-period>
                <jgroups-file>idsk-jgroups.xml</jgroups-file>
                <jgroups-channel>persistence-fs</jgroups-channel>
                <connector-ref>netty-connector</connector-ref>
            </broadcast-group>
        </broadcast-groups>

        <discovery-groups>
            <discovery-group name="my-discovery-group">
                <jgroups-file>idsk-jgroups.xml</jgroups-file>
                <jgroups-channel>persistence-fs</jgroups-channel>
                <refresh-timeout>10000</refresh-timeout>
            </discovery-group>
        </discovery-groups>

        <cluster-connections>
            <cluster-connection name="my-cluster">
                <connector-ref>netty-connector</connector-ref>
                <retry-interval>500</retry-interval>
                <use-duplicate-detection>true</use-duplicate-detection>
                <message-load-balancing>ON_DEMAND</message-load-balancing>
                <max-hops>1</max-hops>
                <discovery-group-ref discovery-group-name="my-discovery-group" />
            </cluster-connection>
        </cluster-connections>

        <cluster-user>admin</cluster-user>
        <cluster-password>admin</cluster-password>

        <diverts>
            <divert name="RF-Transform">
                <routing-name>RFeeds-Transform</routing-name>
                <address>RFTopic</address>
                <forwarding-address>IT-InputQueue</forwarding-address>
                <exclusive>false</exclusive>
            </divert>
            <divert name="RF-Output">
                <routing-name>RFeeds-Output</routing-name>
                <address>RFTopic</address>
                <forwarding-address>T1-InputQueue</forwarding-address>
                <exclusive>false</exclusive>
            </divert>
        </diverts>
        <security-settings>
            <security-setting match="#">
                <permission type="createNonDurableQueue" roles="amq" />
                <permission type="deleteNonDurableQueue" roles="amq" />
                <permission type="createDurableQueue" roles="amq" />
                <permission type="deleteDurableQueue" roles="amq" />
                <permission type="createAddress" roles="amq" />
                <permission type="deleteAddress" roles="amq" />
                <permission type="consume" roles="amq" />
                <permission type="browse" roles="amq" />
                <permission type="send" roles="amq" />
                <!-- we need this otherwise ./artemis data imp wouldn't work -->
                <permission type="manage" roles="amq" />
            </security-setting>
        </security-settings>

        <address-settings>
            <!-- if you define auto-create on certain queues, management has to be 
                auto-create -->
            <address-setting match="activemq.management#">
                <dead-letter-address>DLQ</dead-letter-address>
                <expiry-address>ExpiryQueue</expiry-address>
                <redelivery-delay>60000</redelivery-delay>
                <max-delivery-attempts>5</max-delivery-attempts>
                <!-- with -1 only the global-max-size is in use for limiting -->
                <max-size-bytes>50485760</max-size-bytes>
                <page-size-bytes>10485760</page-size-bytes>
                <message-counter-history-day-limit>10</message-counter-history-day-limit>
                <address-full-policy>PAGE</address-full-policy>
                <auto-create-queues>true</auto-create-queues>
                <auto-create-addresses>false</auto-create-addresses>
            </address-setting>
            <!--default for catch all -->
            <address-setting match="#">
                <dead-letter-address>DLQ</dead-letter-address>
                <expiry-address>ExpiryQueue</expiry-address>
                <redelivery-delay>60000</redelivery-delay>
                <max-delivery-attempts>5</max-delivery-attempts>
                <redistribution-delay>10000</redistribution-delay>
                <!-- with -1 only the global-max-size is in use for limiting -->
                <max-size-bytes>50485760</max-size-bytes>
                <page-size-bytes>10485760</page-size-bytes>
                <message-counter-history-day-limit>10</message-counter-history-day-limit>
                <address-full-policy>PAGE</address-full-policy>
                <auto-create-queues>true</auto-create-queues>
                <auto-create-addresses>false</auto-create-addresses>
            </address-setting>
        </address-settings>

        <addresses>
            <address name="DLQ">
                <anycast>
                    <queue name="DLQ" />
                </anycast>
            </address>
            <address name="ExpiryQueue">
                <anycast>
                    <queue name="ExpiryQueue" />
                </anycast>
            </address>
            <address name="IT-InputQueue">
                <anycast>
                    <queue name="IT-InputQueue" />
                </anycast>
            </address>
            <address name="T1-InputQueue">
                <anycast>
                    <queue name="T1-InputQueue" />
                </anycast>
            </address>
            <address name="RFTopic">
                <multicast />
            </address>
        </addresses>
    </core>
</configuration>

服务器主代理

<?xml version='1.0'?>
<configuration xmlns="urn:activemq"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:xi="http://www.w3.org/2001/XInclude"
    xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">

    <core xmlns="urn:activemq:core"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:activemq:core ">

        <persistence-enabled>true</persistence-enabled>
        <thread-pool-max-size>400</thread-pool-max-size>
        <journal-type>NIO</journal-type>

        <paging-directory>${data.dir}/paging</paging-directory>

        <bindings-directory>${data.dir}/bindings</bindings-directory>

        <journal-directory>${data.dir}/journal</journal-directory>

        <large-messages-directory>${data.dir}/large-messages
        </large-messages-directory>

        <journal-datasync>true</journal-datasync>

        <journal-min-files>20</journal-min-files>

        <journal-pool-files>20</journal-pool-files>

        <journal-file-size>10M</journal-file-size>

        <journal-compact-min-files>30</journal-compact-min-files>

        <journal-buffer-timeout>23480000</journal-buffer-timeout>


        <!-- When using ASYNCIO, this will determine the writing queue depth for 
            libaio. -->
        <journal-max-io>1</journal-max-io>
        <!-- You can verify the network health of a particular NIC by specifying 
            the <network-check-NIC> element. <network-check-NIC>theNicName</network-check-NIC> -->

        <!-- Use this to use an HTTP server to validate the network <network-check-URL-list>http://www.apache.org</network-check-URL-list> -->

        <!-- <network-check-period>10000</network-check-period> -->
        <!-- <network-check-timeout>1000</network-check-timeout> -->

        <!-- this is a comma separated list, no spaces, just DNS or IPs it should 
            accept IPV6 Warning: Make sure you understand your network topology as this 
            is meant to validate if your network is valid. Using IPs that could eventually 
            disappear or be partially visible may defeat the purpose. You can use a list 
            of multiple IPs, and if any successful ping will make the server OK to continue 
            running -->
        <!-- <network-check-list>10.0.0.1</network-check-list> -->

        <!-- use this to customize the ping used for ipv4 addresses -->
        <!-- <network-check-ping-command>ping -c 1 -t %d %s</network-check-ping-command> -->

        <!-- use this to customize the ping used for ipv6 addresses -->
        <!-- <network-check-ping6-command>ping6 -c 1 %2$s</network-check-ping6-command> -->




        <!-- how often we are looking for how many bytes are being used on the 
            disk in ms -->
        <disk-scan-period>5000</disk-scan-period>

        <!-- once the disk hits this limit the system will block, or close the 
            connection in certain protocols that won't support flow control. -->
        <max-disk-usage>90</max-disk-usage>

        <!-- should the broker detect dead locks and other issues -->
        <critical-analyzer>true</critical-analyzer>

        <critical-analyzer-timeout>120000</critical-analyzer-timeout>

        <critical-analyzer-check-period>60000</critical-analyzer-check-period>

        <critical-analyzer-policy>LOG</critical-analyzer-policy>

        <transaction-timeout>1800000</transaction-timeout>
        <!-- the system will enter into page mode once you hit this limit. This 
            is an estimate in bytes of how much the messages are using in memory The 
            system will use half of the available memory (-Xmx) by default for the global-max-size. 
            You may specify a different value here if you need to customize it to your 
            needs. <global-max-size>100Mb</global-max-size> -->
        <connectors>
            <connector name="netty-connector">tcp://localhost:61617</connector>
        </connectors>

        <!-- Acceptors -->
        <acceptors>
            <acceptor name="netty-acceptor">tcp://localhost:61617?anycastPrefix=jms.queue.;multicastPrefix=jms.topic.</acceptor>
        </acceptors>


        <ha-policy>
            <shared-store>
                <master>
                    <failover-on-shutdown>true</failover-on-shutdown>
                </master>
            </shared-store>
        </ha-policy>

        <broadcast-groups>
            <broadcast-group name="my-broadcast-group">
                <broadcast-period>5000</broadcast-period>
                <jgroups-file>idsk-jgroups.xml</jgroups-file>
                <jgroups-channel>persistence-fs</jgroups-channel>
                <connector-ref>netty-connector</connector-ref>
            </broadcast-group>
        </broadcast-groups>

        <discovery-groups>
            <discovery-group name="my-discovery-group">
                <jgroups-file>idsk-jgroups.xml</jgroups-file>
                <jgroups-channel>persistence-fs</jgroups-channel>
                <refresh-timeout>10000</refresh-timeout>
            </discovery-group>
        </discovery-groups>

        <cluster-connections>
            <cluster-connection name="my-cluster">
                <connector-ref>netty-connector</connector-ref>
                <retry-interval>500</retry-interval>
                <use-duplicate-detection>true</use-duplicate-detection>
                <message-load-balancing>ON_DEMAND</message-load-balancing>
                <max-hops>1</max-hops>
                <discovery-group-ref discovery-group-name="my-discovery-group" />
            </cluster-connection>
        </cluster-connections>

        <cluster-user>admin</cluster-user>
        <cluster-password>admin</cluster-password>



        <diverts>
            <divert name="RF-Transform">
                <routing-name>RFeeds-Transform</routing-name>
                <address>RFTopic</address>
                <forwarding-address>IT-InputQueue</forwarding-address>
                <exclusive>false</exclusive>
            </divert>
            <divert name="RF-Output">
                <routing-name>RFeeds-Output</routing-name>
                <address>RFTopic</address>
                <forwarding-address>T1-InputQueue</forwarding-address>
                <exclusive>false</exclusive>
            </divert>
        </diverts>

        <security-settings>
            <security-setting match="#">
                <permission type="createNonDurableQueue" roles="amq" />
                <permission type="deleteNonDurableQueue" roles="amq" />
                <permission type="createDurableQueue" roles="amq" />
                <permission type="deleteDurableQueue" roles="amq" />
                <permission type="createAddress" roles="amq" />
                <permission type="deleteAddress" roles="amq" />
                <permission type="consume" roles="amq" />
                <permission type="browse" roles="amq" />
                <permission type="send" roles="amq" />
                <!-- we need this otherwise ./artemis data imp wouldn't work -->
                <permission type="manage" roles="amq" />
            </security-setting>
        </security-settings>

        <address-settings>
            <!-- if you define auto-create on certain queues, management has to be 
                auto-create -->
            <address-setting match="activemq.management#">
                <dead-letter-address>DLQ</dead-letter-address>
                <expiry-address>ExpiryQueue</expiry-address>
                <redelivery-delay>60000</redelivery-delay>
                <max-delivery-attempts>5</max-delivery-attempts>
                <!-- with -1 only the global-max-size is in use for limiting -->
                <max-size-bytes>50485760</max-size-bytes>
                <page-size-bytes>10485760</page-size-bytes>
                <message-counter-history-day-limit>10</message-counter-history-day-limit>
                <address-full-policy>PAGE</address-full-policy>
                <auto-create-queues>true</auto-create-queues>
                <auto-create-addresses>false</auto-create-addresses>
            </address-setting>
            <!--default for catch all -->
            <address-setting match="#">
                <dead-letter-address>DLQ</dead-letter-address>
                <expiry-address>ExpiryQueue</expiry-address>
                <redelivery-delay>60000</redelivery-delay>
                <max-delivery-attempts>5</max-delivery-attempts>
                <redistribution-delay>10000</redistribution-delay>
                <!-- with -1 only the global-max-size is in use for limiting -->
                <max-size-bytes>50485760</max-size-bytes>
                <page-size-bytes>10485760</page-size-bytes>
                <message-counter-history-day-limit>10</message-counter-history-day-limit>
                <address-full-policy>PAGE</address-full-policy>
                <auto-create-queues>true</auto-create-queues>
                <auto-create-addresses>false</auto-create-addresses>
            </address-setting>          
        </address-settings>

        <addresses>
            <address name="DLQ">
                <anycast>
                    <queue name="DLQ" />
                </anycast>
            </address>
            <address name="ExpiryQueue">
                <anycast>
                    <queue name="ExpiryQueue" />
                </anycast>
            </address>
            <address name="IT-InputQueue">
                <anycast>
                    <queue name="IT-InputQueue" />
                </anycast>
            </address>
            <address name="T1-InputQueue">
                <anycast>
                    <queue name="T1-InputQueue" />
                </anycast>
            </address>
            <address name="RFTopic">
                <multicast />
            </address>
        </addresses>
    </core>
</configuration>

0 个答案:

没有答案