Question

我正在使用spark集成点火查询数据。在出现以下错误之前，程序没有启动很长时间。我检查了ignite的状态，该服务节点存在。

import org.apache.ignite.spark._
import org.apache.ignite.configuration._
import org.apache.ignite.Ignite
import org.apache.ignite.IgniteCache
import org.apache.ignite.Ignition
import org.apache.ignite.scalar.scalar
import org.apache.ignite.scalar.scalar._


// val ic = new IgniteContext(sc, () => new IgniteConfiguration())
val ic = new IgniteContext(sc,"/data/dk/jar/config/default-config.xml")

 val recordCache = ic.ignite.cache[String, AttendRecord]("ATTEND_RECORD_TL")
    var retSet : scala.collection.mutable.Buffer[java.sql.Timestamp] = null
    if(operateTimeSite != null) {
       retSet = recordCache.sql("punchSiteId = ? and operateTime <= ? and operateTime >= ? order by operateTime desc limit 1",punchSiteId, compueteTcTime,operateTimeSite).getAll.map(_.getValue.operateTime) 
    }  else {
       retSet = recordCache.sql("punchSiteId = ? and operateTime <= ?  order by operateTime desc limit 1",punchSiteId, compueteTcTime).getAll.map(_.getValue.operateTime) 
    }
    if(retSet == null || retSet.isEmpty) { 
        if(operateTimeSite !=null) 
            retSet = recordCache.sql("punchSiteId = ? and operateTime >= ?  order by operateTime limit 1",punchSiteId, new Timestamp(operateTimeSite.getTime - 4*3600*1000)).getAll.map(_.getValue.operateTime) 
        else null  
    }

Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start manager: GridManagerAdapter [enabled=true, name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
        at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1791)
        at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:978)
        at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1896)
        at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1648)
        at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1076)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:596)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:536)
        at org.apache.ignite.Ignition.getOrStart(Ignition.java:414)
        ... 37 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start SPI: TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000, marsh=JdkMarshaller [], reconCnt=10, maxAckTimeout=600000, forceSrvMode=false, clientReconnectDisabled=false]
        at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:300)
        at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:842)
        at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1786)
        ... 44 more
Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to connect to cluster, connection failed and failed to reconnect.
        at org.apache.ignite.spi.discovery.tcp.ClientImpl$Reconnector.body(ClientImpl.java:1490)
        at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

Answer 1

您在独立模式下将Ignite集成用于Spark。这意味着，客户端Ignite节点将在内部启动。它将寻找其他节点，并连接到它们。因此，您需要Ignite群集单独运行并可以通过IP（在IP查找器中的IgniteContext的配置中指定的IP）可用。

IP查找程序应包含服务器节点的地址。因此，无需指定内部具有客户端节点的其他Spark执行程序的地址。

“无法连接到群集，连接失败并且无法重新连接” 错误表示，客户端节点无法通过指定的IP查找任何服务器节点。

当启动服务器Ignite节点而不是客户端时，还有一种嵌入式模式，它是独立模式的替代方法。可以使用IgniteContext的另一个构造函数来启用它：

val ic = new IgniteContext(sc,"/data/dk/jar/config/default-config.xml", false)

但是强烈建议不要使用此模式，并且很快将其删除。因此，我建议单独运行Ignite群集。

有关更多信息，请参见以下文档页面：https://apacheignite-fs.readme.io/docs/ignitecontext-igniterdd

PS

您使用multicast IP查找程序，使Ignite扫描已配置的多播组中的所有IP地址。如果您知道服务器节点的所有IP地址，则可以使用TcpDiscoveryVmIpFinder。它将使Ignite实例启动更快。

org.apache.ignite：无法连接到集群，连接失败并且无法重新连接

1 个答案: