如何在远程gremlin-server上为titan / hbase运行gremlin脚本？

时间：2017-06-27 15:14:04

标签： gremlin titan tinkerpop gremlin-server

我使用titan / hbase存储我的数据，Titan和Hbase驻留在远程服务器机器中。我用bin/titan.sh start启动了Gremlin服务器。之后我在服务器机器上运行了以下命令，运行正常：

bin/gremlin.sh 

         \,,,/
         (o o)
-----oOOo-(3)-oOOo-----
plugin activated: aurelius.titan
plugin activated: tinkerpop.server
plugin activated: tinkerpop.utilities
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/titan-1.0.0-hadoop1/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/titan-1.0.0-hadoop1/lib/logback-classic-1.1.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18:32:20 INFO  org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph  - HADOOP_GREMLIN_LIBS is set to: /usr/local/titan-1.0.0-hadoop1/lib
plugin activated: tinkerpop.hadoop
plugin activated: tinkerpop.tinkergraph
gremlin> g = TitanFactory.open("conf/titan-hbase.properties")
==>standardtitangraph[hbase:[192.168.1.65]]
gremlin> gr = g.traversal()
==>graphtraversalsource[standardtitangraph[hbase:[192.168.1.65]], standard]
gremlin> gr.V().count()
18:33:37 WARN  com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx  - Query requires iterating over all vertices [()]. For better performance, use indexes
==>41

现在检查远程连接，我刚刚从客户端计算机运行gremlin.sh并运行

gremlin> :remote connect tinkerpop.server conf/remote.yaml 
g=TitanFactory.open("conf/titan-hbase.properties")

我得到以下信息：

java.lang.RuntimeException: com.thinkaurelius.titan.diskstorage.TemporaryBackendException: Temporary failure in storage backend
    at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.getDeployment(HBaseStoreManager.java:351)
    at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.getFeatures(HBaseStoreManager.java:389)
    at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.<init>(GraphDatabaseConfiguration.java:1321)
    at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:94)
    at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:74)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:78)
    at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:70)
    at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:104)
    at org.apache.tinkerpop.gremlin.server.GraphManager.lambda$new$27(GraphManager.java:50)
    at java.util.LinkedHashMap$LinkedEntrySet.forEach(LinkedHashMap.java:663)
    at org.apache.tinkerpop.gremlin.server.GraphManager.<init>(GraphManager.java:48)
    at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:94)
    at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:88)
    at pluradj.titan.tinkerpop3.example.ServiceTest.startServer(ServiceTest.java:37)
    at pluradj.titan.tinkerpop3.example.ServiceTest.setUp(ServiceTest.java:29)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: com.thinkaurelius.titan.diskstorage.TemporaryBackendException: Temporary failure in storage backend
    at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.ensureTableExists(HBaseStoreManager.java:759)
    at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.getLocalKeyPartition(HBaseStoreManager.java:556)
    at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.getDeployment(HBaseStoreManager.java:347)
    ... 43 more
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=35, exceptions:

    at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:129)
    at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
    at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:264)
    at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:169)
    at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:164)
    at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:107)
    at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:736)
    at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:539)
    at org.apache.hadoop.hbase.catalog.MetaReader.tableExists(MetaReader.java:310)
    at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:279)
    at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:293)
    at com.thinkaurelius.titan.diskstorage.hbase.HBaseAdmin0_98.tableExists(HBaseAdmin0_98.java:93)
    at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.ensureTableExists(HBaseStoreManager.java:753)
    ... 45 more
Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:511)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:481)
    at org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupConnection(RpcClient.java:578)
    at org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:866)
    at org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1536)
    at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1435)
    at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
    at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
    at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:29900)
    at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:302)
    at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:157)
    at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:57)
    at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114)
    ... 57 more

我的titan-hbase.properties是：

storage.backend=hbase
storage.hostname=192.168.1.65
cache.db-cache-time = 180000
cache.db-cache = true
cache.db-cache-clean-wait = 20
cache.db-cache-size = 0.5
gremlin.graph=com.thinkaurelius.titan.core.TtanFactory
storage.hbase.tablename = titan

和remote.yaml是：

hosts: [192.168.1.65]
port: 8182
serializer: { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}

这里出了什么问题？我怎么能做到这一点？

1 个答案:

答案 0 :(得分：0)

您需要做的是使用Titan-HBase属性配置Gremlin Server。您不应该在这种情况下使用titan.sh，因为它会自动启动Cassandra和Elasticsearch节点，因此请确保在继续之前将其关闭bin/titan.sh stop。

  1. 将titan-hbase.properties放在conf/gremlin-server/下。您上面发布的内容看起来不错，但最后一个属性名称应为storage.hbase.table。

  2. 使用您的Titan-HBase配置更新gremlin-server.yaml下的conf/gremlin-server/（默认配置使用Cassandra + Elasticsearch）。另外，请记住，如果要从其他计算机连接到Gremlin Server，则需要将host更新为服务器的IP地址192.168.1.65。

      

    图表：{graph：conf / gremlin-server / titan-hbase.properties}

  3. 使用bin/gremlin-server.sh启动Gremlin服务器。在启动时查看日志以查找错误。如果成功，您应该看到一条INFO消息，其中graph和g已配置并绑定。 g在scripts/empty-sample.groovy脚本中定义，conf/gremlin-server.yaml通过8456 [main] INFO org.apache.tinkerpop.gremlin.server.GremlinServer - Graph [graph] was successfully configured via [conf/gremlin-server/titan-hbase.properties]. 9349 [main] INFO org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor - A GraphTraversalSource is now bound to [g] with graphtraversalsource[standardtitangraph[hbase:[127.0.0.1]], standard]加载。

      

    conf/remote.yaml

  4. 启动Gremlin控制台，使用graph创建一个到Gremlin服务器的远程连接。由于g和:>已经通过服务器配置绑定，因此您无需在控制台中初始化它们。确保在查询开头使用gremlin> :remote connect tinkerpop.server conf/remote.yaml ==>Connected - localhost/127.0.0.1:8182 gremlin> :> g.V().count() ==>12 将命令发送到远程服务器。

    5. 以下是Gremlin控制台会话的示例：

    @Bean
public Job executeConcurrentJob() {
    return jobBuilderFactory.get("executeConcurrentJob")
            .listener(listener())
            .start(step1())
            .build();
}

@Bean
public TaskExecutor taskExecutor() {
    return new SimpleAsyncTaskExecutor();
}

@Bean
public Step step1() {
    return stepBuilderFactory.get("step1").allowStartIfComplete(true)
            .<BatchJob,BatchJob>chunk(10)
            .reader(reader())
            .processor(processor())
            .writer(writer())
            .taskExecutor(taskExecutor())
            .throttleLimit(5)
            .build();
}

    请参阅Apache TinkerPop documentation以从Gremlin控制台连接Gremlin服务器。

相关问题
最新问题