Question

我有一个查询停止/挂起大型参数输入。相同的代码适用于较小的SQL参数输入。代码如下：

subEntries = dataRDD.sql("SELECT v.id,v.sub,v.obj FROM VPRow v JOIN table(id bigint = ?) i ON v.id = i.id",new Object[] {subKeyEntries.toArray()});
LOG.debug("Reading : "+subEntries.count());

请注意，Ignite文档提到输入参数可以是任何大小 - “在这里，您可以提供任意长度的对象数组（Object []）作为参数”。在停滞案例中传递给查询的参数的大小为23641 long个值。

我的弹簧配置文件如下：

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="cacheConfiguration">
            <list>
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <!-- Set a cache name. -->
                    <property name="name" value="dataRDD"/>
                    <!-- Set a cache mode. -->
                    <property name="cacheMode" value="PARTITIONED"/>
                    <!-- Index Integer pairs used in the example. -->
                    <property name="indexedTypes">
                        <list>
                            <value>java.lang.Long</value>
                            <value>sample.VPRow</value>
                        </list>
                    </property>
                    <property name="backups" value="0"/>
                </bean>
            </list>
        </property>
        <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="addresses">
                            <list>
                                <value>[IP1]</value>
                                <value>[...]</value>
                                <value>[IP5]</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>

VPRow类定义如下

public class VPRow implements Serializable {

    @QuerySqlField
    private long id;
    @QuerySqlField
    private String sub;
    @QuerySqlField
    private String obj;

    public VPRow(long id,String sub, String obj) {
        this.id = id;
        this.sub = sub;
        this.obj = obj;
    }
    ...
}

Answer 1

通常数据库对＆＃34; IN＆＃34;操作

您可以将此sql查询分成几部分并同时运行它们。

Answer 2

请参阅此处的第2段：https://apacheignite.readme.io/docs/sql-performance-and-debugging#sql-performance-and-usability-considerations

By default, genfromtxt assumes delimiter=None, meaning that the line is split along white spaces (including tabs) and that consecutive white spaces are considered as a single white space.运算符不使用索引，因此使用像此查询这样的长列表会执行太多扫描。如上所述更改查询应该会有所帮助。

IgniteRDD SQL查询在大输入参数上停滞/缓慢

2 个答案: