将缓存模式设置为REPLICATED时,不会在每个群集节点上复制Apache Ignite缓存数据

时间:2019-01-07 07:55:06

标签: spring-data ignite

我为Apache Ignite创建了两个节点群集的设置。我将此群集用作数据库缓存用例(https://ignite.apache.org/use-cases/caching/database-caching.html)。现在,当我在春季启动应用程序中创建IgniteRepository并从数据库检索条目时,该条目仅缓存在集群中的一个节点中,而我将集群模式设置为REPLICATED,这表示数据被集群中的每个节点缓存( https://apacheignite.readme.io/docs/cache-modes)。请帮助我,我的集群配置有问题还是Ignite拥有这样的实现?

我有一个发现,就是我使用

ignite.cache(“ EmployeesCache”)。loadCache(null);“

在我的春季IgniteConfig中,它在启动时将所有表数据都加载到两个节点上。但是,当我启动具有空缓存的应用程序时,即注释掉上面的行,然后使用

“ getById(id)”

操作,则复制未正确进行。

我的代码如下:

Spring Ignite配置:

@Bean("igniteInstance")
public Ignite startIgniteForDataBaseService() {
    Ignite ignite = Ignition.start("META-INF/EmployeeCluster-client.xml");
   // ignite.cache("EmployeesCache").loadCache(null);
    return ignite;
}

客户端XML:

<?xml version="1.0" encoding="UTF-8"?>

<!-- This file was generated by Ignite Web Console (01/07/2019, 12:15) -->

<beans xmlns="http://www.springframework.org/schema/beans"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:util="http://www.springframework.org/schema/util"
   xsi:schemaLocation="http://www.springframework.org/schema/beans
                       http://www.springframework.org/schema/beans/spring-beans.xsd
                       http://www.springframework.org/schema/util
                       http://www.springframework.org/schema/util/spring-util.xsd">

<!-- Data source beans will be initialized from external properties file. -->
<bean id="dsMySQL_Employees" class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource">
    <property name="URL" value="jdbc:mysql://192.168.138.134:3306/employees"/>
    <property name="user" value="dbadmin"/>
    <property name="password" value="admin"/>
</bean>

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
    <property name="clientMode" value="true"/>
    <property name="igniteInstanceName" value="EmployeeCluster"/>

    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                    <property name="addresses">
                        <list>
                            <value>127.0.0.1:47500..47510</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>

    <property name="cacheConfiguration">
        <list>
            <bean class="org.apache.ignite.configuration.CacheConfiguration">
                <property name="name" value="EmployeesCache"/>
                <property name="cacheMode" value="REPLICATED"/>
                <property name="atomicityMode" value="ATOMIC"/>

                <property name="cacheStoreFactory">
                    <bean class="org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStoreFactory">
                        <property name="dataSourceBean" value="dsMySQL_Employees"/>
                        <property name="dialect">
                            <bean class="org.apache.ignite.cache.store.jdbc.dialect.MySQLDialect">
                            </bean>
                        </property>

                        <property name="types">
                            <list>
                                <bean class="org.apache.ignite.cache.store.jdbc.JdbcType">
                                    <property name="cacheName" value="EmployeesCache"/>
                                    <property name="keyType" value="java.lang.Integer"/>
                                    <property name="valueType" value="com.techacademy.model.Employees"/>
                                    <property name="databaseSchema" value="employees"/>
                                    <property name="databaseTable" value="employees"/>

                                    <property name="keyFields">
                                        <list>
                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.INTEGER"/>
                                                </constructor-arg>
                                                <constructor-arg value="emp_no"/>
                                                <constructor-arg value="int"/>
                                                <constructor-arg value="empNo"/>
                                            </bean>
                                        </list>
                                    </property>

                                    <property name="valueFields">
                                        <list>
                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.INTEGER"/>
                                                </constructor-arg>
                                                <constructor-arg value="emp_no"/>
                                                <constructor-arg value="int"/>
                                                <constructor-arg value="empNo"/>
                                            </bean>
                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.DATE"/>
                                                </constructor-arg>
                                                <constructor-arg value="birth_date"/>
                                                <constructor-arg value="java.sql.Date"/>
                                                <constructor-arg value="birthDate"/>
                                            </bean>

                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.VARCHAR"/>
                                                </constructor-arg>
                                                <constructor-arg value="first_name"/>
                                                <constructor-arg value="java.lang.String"/>
                                                <constructor-arg value="firstName"/>
                                            </bean>

                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.VARCHAR"/>
                                                </constructor-arg>
                                                <constructor-arg value="last_name"/>
                                                <constructor-arg value="java.lang.String"/>
                                                <constructor-arg value="lastName"/>
                                            </bean>

                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.CHAR"/>
                                                </constructor-arg>
                                                <constructor-arg value="gender"/>
                                                <constructor-arg value="java.lang.String"/>
                                                <constructor-arg value="gender"/>
                                            </bean>

                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.DATE"/>
                                                </constructor-arg>
                                                <constructor-arg value="hire_date"/>
                                                <constructor-arg value="java.sql.Date"/>
                                                <constructor-arg value="hireDate"/>
                                            </bean>
                                        </list>
                                    </property>
                                </bean>
                            </list>
                        </property>
                    </bean>
                </property>

                <property name="readThrough" value="true"/>
                <property name="writeThrough" value="true"/>

                <property name="queryEntities">
                    <list>
                        <bean class="org.apache.ignite.cache.QueryEntity">
                            <property name="keyType" value="java.lang.Integer"/>
                            <property name="valueType" value="com.techacademy.model.Employees"/>
                            <property name="keyFieldName" value="empNo"/>

                            <property name="keyFields">
                                <list>
                                    <value>empNo</value>
                                </list>
                            </property>

                            <property name="fields">
                                <map>
                                    <entry key="birthDate" value="java.sql.Date"/>
                                    <entry key="firstName" value="java.lang.String"/>
                                    <entry key="lastName" value="java.lang.String"/>
                                    <entry key="gender" value="java.lang.String"/>
                                    <entry key="hireDate" value="java.sql.Date"/>
                                    <entry key="empNo" value="java.lang.Integer"/>
                                </map>
                            </property>

                            <property name="aliases">
                                <map>
                                    <entry key="empNo" value="emp_no"/>
                                    <entry key="birthDate" value="birth_date"/>
                                    <entry key="firstName" value="first_name"/>
                                    <entry key="lastName" value="last_name"/>
                                    <entry key="hireDate" value="hire_date"/>
                                </map>
                            </property>
                        </bean>
                    </list>
                </property>
            </bean>
        </list>
    </property>
</bean>

服务器XML:

<?xml version="1.0" encoding="UTF-8"?>

<!-- This file was generated by Ignite Web Console (01/07/2019, 12:15) -->

<beans xmlns="http://www.springframework.org/schema/beans"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:util="http://www.springframework.org/schema/util"
   xsi:schemaLocation="http://www.springframework.org/schema/beans
                       http://www.springframework.org/schema/beans/spring-beans.xsd
                       http://www.springframework.org/schema/util
                       http://www.springframework.org/schema/util/spring-util.xsd">

<!-- Data source beans will be initialized from external properties file. -->
<bean id="dsMySQL_Employees" class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource">
    <property name="URL" value="jdbc:mysql://192.168.138.134:3306/employees"/>
    <property name="user" value="dbadmin"/>
    <property name="password" value="admin"/>
</bean>

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
    <property name="igniteInstanceName" value="EmployeeCluster"/>

    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                    <property name="addresses">
                        <list>
                            <value>127.0.0.1:47500..47510</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>

    <property name="cacheConfiguration">
        <list>
            <bean class="org.apache.ignite.configuration.CacheConfiguration">
                <property name="name" value="EmployeesCache"/>
                <property name="cacheMode" value="REPLICATED"/>
                <property name="atomicityMode" value="ATOMIC"/>

                <property name="cacheStoreFactory">
                    <bean class="org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStoreFactory">
                        <property name="dataSourceBean" value="dsMySQL_Employees"/>
                        <property name="dialect">
                            <bean class="org.apache.ignite.cache.store.jdbc.dialect.MySQLDialect">
                            </bean>
                        </property>

                        <property name="types">
                            <list>
                                <bean class="org.apache.ignite.cache.store.jdbc.JdbcType">
                                    <property name="cacheName" value="EmployeesCache"/>
                                    <property name="keyType" value="java.lang.Integer"/>
                                    <property name="valueType" value="com.techacademy.model.Employees"/>
                                    <property name="databaseSchema" value="employees"/>
                                    <property name="databaseTable" value="employees"/>

                                    <property name="keyFields">
                                        <list>
                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.INTEGER"/>
                                                </constructor-arg>
                                                <constructor-arg value="emp_no"/>
                                                <constructor-arg value="int"/>
                                                <constructor-arg value="empNo"/>
                                            </bean>
                                        </list>
                                    </property>

                                    <property name="valueFields">
                                        <list>
                                         <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.INTEGER"/>
                                                </constructor-arg>
                                                <constructor-arg value="emp_no"/>
                                                <constructor-arg value="int"/>
                                                <constructor-arg value="empNo"/>
                                            </bean>
                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.DATE"/>
                                                </constructor-arg>
                                                <constructor-arg value="birth_date"/>
                                                <constructor-arg value="java.sql.Date"/>
                                                <constructor-arg value="birthDate"/>
                                            </bean>

                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.VARCHAR"/>
                                                </constructor-arg>
                                                <constructor-arg value="first_name"/>
                                                <constructor-arg value="java.lang.String"/>
                                                <constructor-arg value="firstName"/>
                                            </bean>

                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.VARCHAR"/>
                                                </constructor-arg>
                                                <constructor-arg value="last_name"/>
                                                <constructor-arg value="java.lang.String"/>
                                                <constructor-arg value="lastName"/>
                                            </bean>

                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.CHAR"/>
                                                </constructor-arg>
                                                <constructor-arg value="gender"/>
                                                <constructor-arg value="java.lang.String"/>
                                                <constructor-arg value="gender"/>
                                            </bean>

                                            <bean class="org.apache.ignite.cache.store.jdbc.JdbcTypeField">
                                                <constructor-arg>
                                                    <util:constant static-field="java.sql.Types.DATE"/>
                                                </constructor-arg>
                                                <constructor-arg value="hire_date"/>
                                                <constructor-arg value="java.sql.Date"/>
                                                <constructor-arg value="hireDate"/>
                                            </bean>
                                        </list>
                                    </property>
                                </bean>
                            </list>
                        </property>
                    </bean>
                </property>

                <property name="readThrough" value="true"/>
                <property name="writeThrough" value="true"/>

                <property name="queryEntities">
                    <list>
                        <bean class="org.apache.ignite.cache.QueryEntity">
                            <property name="keyType" value="java.lang.Integer"/>
                            <property name="valueType" value="com.techacademy.model.Employees"/>
                            <property name="keyFieldName" value="empNo"/>

                            <property name="keyFields">
                                <list>
                                    <value>empNo</value>
                                </list>
                            </property>

                            <property name="fields">
                                <map>
                                    <entry key="birthDate" value="java.sql.Date"/>
                                    <entry key="firstName" value="java.lang.String"/>
                                    <entry key="lastName" value="java.lang.String"/>
                                    <entry key="gender" value="java.lang.String"/>
                                    <entry key="hireDate" value="java.sql.Date"/>
                                    <entry key="empNo" value="java.lang.Integer"/>
                                </map>
                            </property>

                            <property name="aliases">
                                <map>
                                    <entry key="empNo" value="emp_no"/>
                                    <entry key="birthDate" value="birth_date"/>
                                    <entry key="firstName" value="first_name"/>
                                    <entry key="lastName" value="last_name"/>
                                    <entry key="hireDate" value="hire_date"/>
                                </map>
                            </property>
                        </bean>
                    </list>
                </property>
            </bean>
        </list>
    </property>
</bean>

Ingite存储库:

@RepositoryConfig(cacheName = "EmployeesCache")
public interface EmployeeRepository extends IgniteRepository<Employees,Integer> {
}

其他控制器:

  @RestController
  public class RetrieveEmployeeController {

    @Autowired
    private EmployeeRepository employeeRepository;

    @GetMapping("/employee/{id}")
    public Employees getEmployeeDetails(@PathVariable("id") Integer id){
        return  employeeRepository.findById(id).orElseGet(null);
    }
  }

我期望当我使用findById时,数据将在两个节点上复制,但不会在其他节点上复制。当前的输出缓存统计信息如下:

visor> cache -a
Time of the snapshot: 2019-01-07 13:22:49
+================================================================================================================================================================+
|       Name(@)       |    Mode    | Nodes | Total entries (Heap / Off-heap) | Primary entries (Heap / Off-heap) |   Hits    |  Misses   |   Reads   |  Writes   |
+================================================================================================================================================================+
| EmployeesCache(@c0) | REPLICATED | 2     | 1 (0 / 1)                       | min: 0 (0 / 0)                    | min: 0    | min: 0    | min: 0    | min: 0    |
|                     |            |       |                                 | avg: 0.50 (0.00 / 0.50)           | avg: 0.00 | avg: 0.00 | avg: 0.00 | avg: 0.00 |
|                     |            |       |                                 | max: 1 (0 / 1)                    | max: 0    | max: 0    | max: 0    | max: 0    |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------+

Cache 'EmployeesCache(@c0)':
+-------------------------------------------------------+
| Name(@)                         | EmployeesCache(@c0) |
| Total entries (Heap / Off-heap) | 1 (0 / 1)           |
| Nodes                           | 2                   |
| Total size Min/Avg/Max          | 0 / 0.50 / 1        |
|   Heap size Min/Avg/Max         | 0 / 0.00 / 0        |
|   Off-heap size Min/Avg/Max     | 0 / 0.50 / 1        |
+-------------------------------------------------------+

Nodes for: EmployeesCache(@c0)
+====================================================================================================================+
|       Node ID8(@), IP        | CPUs | Heap Used | CPU Load |   Up Time    | Size (Primary / Backup)  | Hi/Mi/Rd/Wr |
+====================================================================================================================+
| 90CD3FAD(@n1), 192.168.137.1 | 8    | 5.49 %    | 0.07 %   | 00:50:35.127 | Total: 1 (1 / 0)         | Hi: 0       |
|                              |      |           |          |              |   Heap: 0 (0 / <n/a>)    | Mi: 0       |
|                              |      |           |          |              |   Off-Heap: 1 (1 / 0)    | Rd: 0       |
|                              |      |           |          |              |   Off-Heap Memory: <n/a> | Wr: 0       |
+------------------------------+------+-----------+----------+--------------+--------------------------+-------------+
| AF07CA98(@n0), 192.168.137.1 | 8    | 28.71 %   | 0.00 %   | 00:50:40.230 | Total: 0 (0 / 0)         | Hi: 0       |
|                              |      |           |          |              |   Heap: 0 (0 / <n/a>)    | Mi: 0       |
|                              |      |           |          |              |   Off-Heap: 0 (0 / 0)    | Rd: 0       |
|                              |      |           |          |              |   Off-Heap Memory: 0     | Wr: 0       |
+--------------------------------------------------------------------------------------------------------------------+
'Hi' - Number of cache hits.
'Mi' - Number of cache misses.
'Rd' - number of cache reads.
'Wr' - Number of cache writes.

Aggregated queries metrics:
  Minimum execution time: 00:00:00.000
  Maximum execution time: 00:00:00.000
  Average execution time: 00:00:00.000
  Total number of executions: 0
  Total number of failures:   0

1 个答案:

答案 0 :(得分:0)

我能够在master分支上重现此问题,并创建了一张Jira票证来修复此IGNITE-10950 Lost backup entries when using spring data and cache store。这样,您就可以跟踪解决此问题的进度。

目前,作为一种解决方法,您可以使用IgniteCache#get代替IgniteRepository#findById