将Apache Cassandra与Apache Ignite集成

时间:2018-04-24 07:47:25

标签: cassandra ignite cassandra-3.0 sqlline

我正在尝试将Apache Ignite与Apache Cassandra(3.11.2)集成,因为我想使用Ignite来缓存我现有的Cassandra数据库中存在的数据。

在浏览完在线资源后,我到目前为止做了以下工作:

  1. 已下载Apache Ignite
  2. 将“libs / optional /”中的所有文件夹复制到“libs /”(我不知道Cassandra需要哪些文件夹)。
  3. 在config文件夹中创建了3个xmls,即“cassandra-config.xml”,“connection-settings.xml”和“persistance-settings.xml”。目前我正在为Cassandra和Ignite使用相同的节点(172.16.129.68)。
  4. 卡桑德拉-config.xml中

    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="
            http://www.springframework.org/schema/beans
            http://www.springframework.org/schema/beans/spring-beans.xsd">
    <!-- Cassandra connection settings -->
    <import resource="connection-settings.xml" />
    
    <!-- Persistence settings for 'cache1' -->
    <bean id="cache1_persistence_settings" class="org.apache.ignite.cache.store.cassandra.persistence.KeyValuePersistenceSettings">
            <constructor-arg type="org.springframework.core.io.Resource" value="file:/home/cass/apache_ignite/apache-ignite-fabric-2.4.0-bin/config/persistance-settings.xml" />
    </bean>
    <!-- Ignite configuration -->
    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="cacheConfiguration">
            <list>
                <!-- Configuring persistence for "cache1" cache -->
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="cache1"/>
                    <property name="readThrough" value="true"/>
                    <property name="writeThrough" value="true"/>
                    <property name="writeBehindEnabled" value="true"/>
                    <property name="cacheStoreFactory">
                        <bean class="org.apache.ignite.cache.store.cassandra.CassandraCacheStoreFactory">
                            <property name="dataSourceBean" value="cassandraAdminDataSource"/>
                            <property name="persistenceSettingsBean" value="cache1_persistence_settings"/>
                        </bean>
                    </property>
                </bean>
            </list>
        </property>
        <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <!--
                        Ignite provides several options for automatic discovery that can be used
                        instead os static IP based discovery. For information on all options refer
                        to our documentation: http://apacheignite.readme.io/docs/cluster-config
                    -->
                    <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
                    <!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">-->
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="addresses">
                            <list>
                                <!-- In distributed environment, replace with actual host IP address. -->
                                <value>172.16.129.68:47500..47509</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
    

    连接的settings.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="
            http://www.springframework.org/schema/beans
            http://www.springframework.org/schema/beans/spring-beans.xsd">
    
    <bean id="loadBalancingPolicy" class="com.datastax.driver.core.policies.TokenAwarePolicy">
        <constructor-arg type="com.datastax.driver.core.policies.LoadBalancingPolicy">
            <bean class="com.datastax.driver.core.policies.RoundRobinPolicy"/>
        </constructor-arg>
    </bean>
    
    <bean id="cassandraAdminDataSource" class="org.apache.ignite.cache.store.cassandra.datasource.DataSource">
        <property name="port" value="9042"/>
        <property name="contactPoints" value="172.16.129.68"/>
        <property name="readConsistency" value="ONE"/>
        <property name="writeConsistency" value="ONE"/>
        <property name="loadBalancingPolicy" ref="loadBalancingPolicy"/>
    </bean>
    

    持久性-settings.xml中

    <persistence keyspace="test" table="epc_table">
        <keyPersistence class="java.lang.String" strategy="PRIMITIVE" column="imsi"/>
        <valuePersistence strategy="BLOB"/>
    </persistence>
    
    1. 我运行以下命令从bin文件夹启动Ignite。

      ignite.sh ../ config / cassandra-config.xml

    2. 现在,我想通过sqlline查看cassandra表。我尝试了以下内容:

        

      ./ sqlline.sh -u jdbc:cassandra://172.16.129.68:9042 / test //(test是键空间的名称)

      我得到以下输出:

      No known driver to handle "jdbc:cassandra://172.16.129.68:9042/test". Searching for known drivers...
      java.lang.NullPointerException
      sqlline version 1.3.0
      0: jdbc:cassandra://172.16.129.68:9042/test>
      

      我也试过了:

      ./sqlline.sh -u jdbc:ignite:thin://172.16.129.68
      

      但是当我使用“!tables”时,我无法看到任何表格。

      究竟错过了什么?如何使用sqlline访问/修改Cassandra中的表?

      操作系统:RHEL 6.5

1 个答案:

答案 0 :(得分:3)

Apache Ignite是一个键值数据库,默认情况下不会创建任何可以使用JDBC连接器查看的表。 CacheStore是一种将Ignite与外部数据库或任何其他存储集成的方法,它将数据作为键值对加载。

在您的配置中,您说Ignite要在Cassandra中存储和加载条目,但仍然Ignite不知道条目结构(BTW Ignite实际上并不关心哪些对象被放入其中)。

为了能够列出表并对其进行查询,您需要创建表。为此,您需要在/ lib目录中使用ignite-indexing,如果您有注释的POJO,则设置QueryEntityindexed types。以下是此类配置的示例:

<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="mycache"/>
<!-- Configure query entities -->
<property name="queryEntities">
    <list>
        <bean class="org.apache.ignite.cache.QueryEntity">
            <property name="keyType" value="java.lang.Long"/>
            <property name="valueType" value="org.apache.ignite.examples.Person"/>

            <property name="fields">
                <map>
                    <entry key="id" value="java.lang.Long"/>
                    <entry key="orgId" value="java.lang.Long"/>
                    <entry key="firstName" value="java.lang.String"/>
                    <entry key="lastName" value="java.lang.String"/>
                    <entry key="resume" value="java.lang.String"/>
                    <entry key="salary" value="java.lang.Double"/>
                </map>
            </property>

            <property name="indexes">
                <list>
                    <bean class="org.apache.ignite.cache.QueryIndex">
                        <constructor-arg value="id"/>
                    </bean>
                    <bean class="org.apache.ignite.cache.QueryIndex">
                        <constructor-arg value="orgId"/>
                    </bean>
                    <bean class="org.apache.ignite.cache.QueryIndex">
                        <constructor-arg value="salary"/>
                    </bean>
                </list>
            </property>
        </bean>
    </list>
</property>

如果您配置了该功能,您将能够通过SQLine登记和查询这些表。 (请注意,您无法查询未加载到Ignite中的数据。要加载它们,您可以使用IgniteCache.get()启用readThrough选项或IgniteCache.loadCache()加载Cassandra表中的所有内容。

要使用JDBC查询Cassandra,您需要一个JDBC驱动程序,请尝试,例如DBSchema