通过GAR文件调用远程任务

时间：2016-01-12 12:26:42

标签： ignite

我使用ignite.xml bean从我的一个节点

中的目录加载GAR文件

我有以下GAR <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:util="http://www.springframework.org/schema/util" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-2.5.xsd"> <util:list id="myList" value-type="java.lang.String"> <value>myproject.HelloWorldTask</value> <value>myproject.SimpleTask</value> </util:list> </beans> 文件（花了我一段时间才弄清楚这个，顺便说一句？）

package myproject;

public class HelloWorldTask extends ComputeTaskAdapter<String, Integer> {

    static {
        System.out.println("TheGlue: Loading HelloWorldTask ");
    }

    public HelloWorldTask() {
    }

    @Nullable
    @Override
    public Map<? extends ComputeJob, ClusterNode> map(List<ClusterNode> nodes, @Nullable String arg) throws IgniteException {
        System.out.println("Hello from GAR file");
        return null;  //To change body of implemented methods use File | Settings | File Templates.
    }

    @Nullable
    @Override
    public Integer reduce(List<ComputeJobResult> results) throws IgniteException {
        return null;  //To change body of implemented methods use File | Settings | File Templates.
    }
}

HelloWorldTask：

package myproject;

@ComputeTaskName("SimpleTaskName")
public class SimpleTask implements ComputeTask<String, Integer> {


    static {
        System.out.println("Loading SimpleTask");
    }

    public SimpleTask() {
    }

    @Override
    public Map<? extends ComputeJob, ClusterNode> map(List<ClusterNode> subgrid, String arg) throws IgniteException {
        System.out.println("Computing Job in SimpleTask ");
        return null;  //To change body of implemented methods use File | Settings | File Templates.
    }

    @Override
    public ComputeJobResultPolicy result(ComputeJobResult res, List<ComputeJobResult> rcvd) throws IgniteException {
        return null;  //To change body of implemented methods use File | Settings | File Templates.
    }

    @Override
    public Integer reduce(List<ComputeJobResult> results) throws IgniteException {
        return null;  //To change body of implemented methods use File | Settings | File Templates.
    }
}

SimpleTask：

GridUriDeploymentSpringDocument

Ignite可以找到2个类（通过GridUriDeploymentFileProcessor和public class _03GarTest { public static void main(String[] args) { System.out.println("Start urideployment test"); IgniteConfiguration cfg = new IgniteConfiguration(); cfg.setPeerClassLoadingEnabled(true); //needs to be the same as in the XML for the server cfg.setClientMode(true); try(Ignite ignite = Ignition.start(cfg)) { ignite.compute(ignite.cluster().forRemotes()).execute("SimpleTaskName", null); } } } 进行调试，找到并加载它们）。 Ignite说它找到了GAR，但据我所知，这些类没有实例化。日志文件中没有错误，也没有指示任务已部署。

我正在尝试在未部署GAR文件的节点（即群集的客户端节点）上执行以下代码，但是没有在群集上执行任务：

_03GarTest

执行SimpleTaskName类的日志文件（如果我使用“myproject.SimpleTaskName”或“Exception in thread "main" class org.apache.ignite.IgniteDeploymentException: Unknown task name or failed to auto-deploy task (was task (re|un)deployed?): SimpleTaskName at org.apache.ignite.internal.util.IgniteUtils$7.apply(IgniteUtils.java:761) at org.apache.ignite.internal.util.IgniteUtils$7.apply(IgniteUtils.java:759) at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:877) at org.apache.ignite.internal.IgniteComputeImpl.execute(IgniteComputeImpl.java:154) at _03GarTest.main(_03GarTest.java:55) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) Caused by: class org.apache.ignite.internal.IgniteDeploymentCheckedException: Unknown task name or failed to auto-deploy task (was task (re|un)deployed?): SimpleTaskName at org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:515) at org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:447) at org.apache.ignite.internal.IgniteComputeImpl.execute(IgniteComputeImpl.java:151) ... 6 more ”运行，则相同），在客户机节点上转储以下堆栈跟踪：

[13:13:33,057][INFO][disco-event-worker-#48%null%][GridDiscoveryManager] Added new node to topology: TcpDiscoveryNode [id=b70dce5e-c0fd-4ffe-8dc2-b72b18db76da, addrs=[0:0:0:0:0:0:0:1, 10.1.26.59, 127.0.0.1, 192.168.8.103, 192.168.99.1], sockAddrs=[/192.168.8.103:0, /0:0:0:0:0:0:0:1:0, /192.168.99.1:0, /10.1.26.59:0, /10.1.26.59:0, /127.0.0.1:0, /192.168.8.103:0, /192.168.99.1:0], discPort=0, order=12, intOrder=7, lastExchangeTime=1452600812926, loc=false, ver=1.5.0#20151229-sha1:f1f8cda2, isClient=true]
    [13:13:33,063][INFO][disco-event-worker-#48%null%][GridDiscoveryManager] Topology snapshot [ver=12, servers=1, clients=1, CPUs=8, heap=1.5GB]
    [13:13:33,085][WARNING][disco-event-worker-#48%null%][CourtesyConfigNotice] 

    >>> +-------------------------------------------------------------------+
    >>> + Courtesy notice that joining node has inconsistent configuration. +
    >>> + Ignore this message if you are sure that this is done on purpose. +
    >>> +-------------------------------------------------------------------+
    >>> Remote Node ID: B70DCE5E-C0FD-4FFE-8DC2-B72B18DB76DA
    >>> Remote SPI with the same name is not configured: UriDeploymentSpi
    >>> => Local node:  o.a.i.spi.deployment.uri.UriDeploymentSpi

    [13:13:33,103][INFO][exchange-worker-#51%null%][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=12, minorTopVer=0], evt=NODE_JOINED, node=b70dce5e-c0fd-4ffe-8dc2-b72b18db76da]
    [13:13:33,907][INFO][disco-event-worker-#48%null%][GridDiscoveryManager] Node left topology: TcpDiscoveryNode [id=b70dce5e-c0fd-4ffe-8dc2-b72b18db76da, addrs=[0:0:0:0:0:0:0:1, 10.1.26.59, 127.0.0.1, 192.168.8.103, 192.168.99.1], sockAddrs=[/192.168.8.103:0, /0:0:0:0:0:0:0:1:0, /192.168.99.1:0, /10.1.26.59:0, /10.1.26.59:0, /127.0.0.1:0, /192.168.8.103:0, /192.168.99.1:0], discPort=0, order=12, intOrder=7, lastExchangeTime=1452600812926, loc=false, ver=1.5.0#20151229-sha1:f1f8cda2, isClient=true]
    [13:13:33,908][INFO][disco-event-worker-#48%null%][GridDiscoveryManager] Topology snapshot [ver=13, servers=1, clients=0, CPUs=8, heap=1.0GB]
    [13:13:33,918][INFO][exchange-worker-#51%null%][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=13, minorTopVer=0], evt=NODE_LEFT, node=b70dce5e-c0fd-4ffe-8dc2-b72b18db76da]
    [13:14:03,193][INFO][grid-timeout-worker-#33%null%][IgniteKernal]

在服务器上，生成以下日志：

        System.out.println("Start urideployment test");

        IgniteConfiguration cfg = new IgniteConfiguration();
        cfg.setPeerClassLoadingEnabled(true); //needs to be the same as in the XML for the server
        cfg.setClientMode(true);

        UriDeploymentSpi deploymentSpi = new UriDeploymentSpi();

        deploymentSpi.setUriList(Arrays.asList("file:///Users/sbeaupre/Dropbox/prorabel/Projects/IgniteTests/ignite/gar"));

        cfg.setDeploymentSpi(deploymentSpi);

        try(Ignite ignite = Ignition.start(cfg)) {
...

关于如何调用通过另一个节点上的GAR文件部署的任务的任何想法？

---- ---- UPDATE

正如其中一个答案中所建议的那样，我在客户端

中添加了以下代码 
Jan 14, 2016 5:42:23 PM org.apache.ignite.logger.java.JavaLogger info
INFO: Topology snapshot [ver=4, servers=1, clients=1, CPUs=8, heap=1.5GB]
Jan 14, 2016 5:42:23 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from resource loaded from byte array
Jan 14, 2016 5:42:23 PM org.apache.ignite.logger.java.JavaLogger info
INFO: User version is not explicitly defined (will use default version) [file=META-INF/ignite.xml, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]]]
Jan 14, 2016 5:42:23 PM org.apache.ignite.logger.java.JavaLogger info
INFO: Task locally deployed: class myproject.SimpleTask
Loading SimpleTask
Computing Job in SimpleTask 
Jan 14, 2016 5:42:23 PM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Failed to map task jobs to nodes: GridTaskSessionImpl [taskName=SimpleTaskName, dep=GridDeployment [ts=1452789743727, depMode=SHARED, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]], clsLdrId=cc234014251-301a4cb8-6fc7-4aa9-b050-3083183f4cd0, userVer=0, loc=true, sampleClsName=myproject.SimpleTask, pendingUndeploy=false, undeployed=false, usage=1], taskClsName=myproject.SimpleTask, sesId=bc234014251-301a4cb8-6fc7-4aa9-b050-3083183f4cd0, startTime=1452789743638, endTime=9223372036854775807, taskNodeId=301a4cb8-6fc7-4aa9-b050-3083183f4cd0, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]], closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, subjId=301a4cb8-6fc7-4aa9-b050-3083183f4cd0, mapFut=IgniteFuture [orig=GridFutureAdapter [resFlag=0, res=null, startTime=1452789743739, endTime=0, ignoreInterrupts=false, lsnr=null, state=INIT]]]
class org.apache.ignite.IgniteCheckedException: Task map operation produced no mapped jobs: GridTaskSessionImpl [taskName=SimpleTaskName, dep=GridDeployment [ts=1452789743727, depMode=SHARED, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]], clsLdrId=cc234014251-301a4cb8-6fc7-4aa9-b050-3083183f4cd0, userVer=0, loc=true, sampleClsName=myproject.SimpleTask, pendingUndeploy=false, undeployed=false, usage=1], taskClsName=myproject.SimpleTask, sesId=bc234014251-301a4cb8-6fc7-4aa9-b050-3083183f4cd0, startTime=1452789743638, endTime=9223372036854775807, taskNodeId=301a4cb8-6fc7-4aa9-b050-3083183f4cd0, clsLdr=GridUriDeploymentClassLoader [urls=[file:/var/folders/t3/595tz_px2j9__wl37f0b5nw40000gn/T/gg.uri.deployment.tmp/301a4cb8-6fc7-4aa9-b050-3083183f4cd0/dirzip_Archive8035449106801616883.gar/]], closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, subjId=301a4cb8-6fc7-4aa9-b050-3083183f4cd0, mapFut=IgniteFuture [orig=GridFutureAdapter [resFlag=0, res=null, startTime=1452789743739, endTime=0, ignoreInterrupts=false, lsnr=null, state=INIT]]]
    at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:497)
    at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
    at org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:678)
    at org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:447)
    at org.apache.ignite.internal.IgniteComputeImpl.execute(IgniteComputeImpl.java:151)
    at _03GarTest.main(_03GarTest.java:55)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

但这也不起作用，我在客户端节点上跟踪堆栈跟踪，而在服务器节点上没有任何内容：

{{1}}

1 个答案:

答案 0 :(得分：2)

斯文，

您应该在客户端节点上配置URI部署SPI，以使GAR部署正常工作。

当您致电compute.execute("taskName");时，必须在客户端本地完成许多事情，然后才能将第一个请求发送到拓扑中的任何节点并在结果开始返回之后。至少，Ignite应该能够获得映射的作业，并能够处理来自所有远程作业的结果并减少所有结果 - 请参阅ComputeTask.map()和ComputeTask.result()以及ComputeTask.reduce()。因此，您应该能够在客户端节点上实例化任务，这就是为什么您应该有可用的任务类。

我认为在客户端节点上配置URI部署后，您的代码应该可以正常工作。

如果您需要任何其他信息，请在此发表评论。

谢谢！

2016年1月18日更新

这是针对问题更新的更新。

请注意，有问题的任务从map（）方法返回null，这是非法的。您可以在二进制版本中引用org.apache.ignite.examples.computegrid.ComputeTaskMapExample或直接通过https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob;f=examples/src/main/java/org/apache/ignite/examples/computegrid/ComputeTaskMapExample.java;h=3de5293a814e527b57e3984f6d3ab96bb1b62daf;hb=HEAD

相关问题
最新问题