Question

我有一个Flask Web应用程序，我想在其中保持与AWS Neptune图形数据库的持久连接。建立此连接的方法如下：

from gremlin_python.process.anonymous_traversal import traversal
from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection

neptune_endpt = 'db-instance-x.xxxxxxxxxx.xx-xxxxx-x.neptune.amazonaws.com'
remoteConn = DriverRemoteConnection(f'wss://{neptune_endpt}:8182/gremlin','g')
self.g = traversal().withRemote(remoteConn)

我面临的问题是，如果闲置时连接会自动断开，并且我无法找到一种方法来检测连接是否断开（以便我可以使用上面的代码段重新连接）。

我也看到过类似的问题：Gremlin server withRemote connection closed - how to reconnect automatically?，但是这个问题也没有解决方案。这个similar question也没有答案。

我尝试了以下两种解决方案（两者均无效）：

我将Web应用程序设置在四名Gunicorn工作人员之后，超时时间为100秒，希望工作人员重新启动可以解决Gremlin超时问题。
我尝试捕获异常以检测连接是否断开。每次我使用self.g对图形进行遍历时，我都会尝试“刷新”连接，这是我的意思：

def _refresh_neptune(self):
    try:
        self.g = traversal().withRemote(self.conn)
    except:
        self.conn = DriverRemoteConnection(f'wss://{neptune_endpt}:8182/gremlin','g')
        self.g = traversal().withRemote(self.conn)

此处self.conn初始化为：

self.conn = DriverRemoteConnection(f'wss://{neptune_endpt}:8182/gremlin','g')

有什么办法可以解决此连接错误？

谢谢

更新：添加了以下错误消息：

  File "/home/ubuntu/.virtualenvs/rundev/lib/python3.6/site-packages/gremlin_python/process/traversal.py
", line 58, in toList
    return list(iter(self))
  File "/home/ubuntu/.virtualenvs/rundev/lib/python3.6/site-packages/gremlin_python/process/traversal.py
", line 48, in __next__
    self.traversal_strategies.apply_strategies(self)
  File "/home/ubuntu/.virtualenvs/rundev/lib/python3.6/site-packages/gremlin_python/process/traversal.py
", line 573, in apply_strategies
    traversal_strategy.apply(traversal)
  File "/home/ubuntu/.virtualenvs/rundev/lib/python3.6/site-packages/gremlin_python/driver/remote_connec
tion.py", line 149, in apply
    remote_traversal = self.remote_connection.submit(traversal.bytecode)
  File "/home/ubuntu/.virtualenvs/rundev/lib/python3.6/site-packages/gremlin_python/driver/driver_remote
_connection.py", line 56, in submit
    results = result_set.all().result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/home/ubuntu/.virtualenvs/rundev/lib/python3.6/site-packages/gremlin_python/driver/resultset.py"
, line 90, in cb
    f.result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/ubuntu/.virtualenvs/rundev/lib/python3.6/site-packages/gremlin_python/driver/connection.py
", line 83, in _receive
    status_code = self._protocol.data_received(data, self._results)
  File "/home/ubuntu/.virtualenvs/rundev/lib/python3.6/site-packages/gremlin_python/driver/protocol.py",
 line 81, in data_received
    'message': 'Server disconnected - please try to reconnect', 'attributes': {}})
gremlin_python.driver.protocol.GremlinServerError: 500: Server disconnected - please try to reconnect

Answer 1

我不确定这是否是解决此问题的最佳方法，但我也使用gremlin-python和Neptune，但遇到了同样的问题。我通过实现可以提供给DriverRemoteConnection的传输来解决此问题。

DriverRemoteConnection(
    url=endpoint,
    traversal_source=self._traversal_source,
    transport_factory=Transport
)

gremlin-python会在发生异常时将连接返回到池，而在关闭连接时返回的异常是GremlinServerError，其他错误也会引发该异常。

gremlin_python/driver/connection.py#L69- gremlin_python/driver/protocol.py#L80

自定义传输与gremlin-python的TornadoTransport相同，但读写方法扩展到：

如果关闭了Web套接字客户端，请重新打开关闭的连接
如果Web套接字客户端从read_message返回None，则引发StreamClosedError

添加到池中的死连接可以重新挂起，然后可以处理StreamClosedError以应用一些重试逻辑。我通过重写DriverRemoteConnection中的commit和submitAsync方法来做到这一点，但是您可以在任何地方捕获并重试。

class Transport(AbstractBaseTransport):
    def __init__(self):
        self._ws = None
        self._loop = ioloop.IOLoop(make_current=False)
        self._url = None

        # Because the transport will try to reopen the underlying ws connection
        # track if the closed() method has been called to prevent the transport
        # from reopening.
        self._explicit_closed = True

    @property
    def closed(self):
        return not self._ws.protocol

    def connect(self, url, headers=None):
        self._forced_closed = False

        # Set the endpoint URL
        self._url = httpclient.HTTPRequest(url, headers=headers) if headers else url

        # Open the connection
        self._connect()

    def write(self, message):
        # Before writing, try to ensure that the connection is open.
        if self.closed:
            self._connect()

        self._loop.run_sync(lambda: self._ws.write_message(message, binary=True))

    def read(self):
        result = self._loop.run_sync(self._ws.read_message)

        # If the read call returns None, the stream has closed.
        if result is None:
            self._ws.close()  # Ensure we close the stream
            raise StreamClosedError()

        return result

    def close(self):
        self._ws.close()
        self._loop.close()
        self._explicit_closed = True

    def _connect(self):
        # If close() was called explicitly on the transport, don't allow
        # subsequent calls to write() to reopen the connection.
        if self._explicit_closed:
            raise TransportClosedError(
                "Transport has been closed and can not be reopened."
            )

        # Check if the ws is closed, if it is not, close it.
        if self._ws and not self.closed:
            self._ws.close()

        # Open the ws connection
        self._ws = self._loop.run_sync(
            lambda: websocket.websocket_connect(url=self._url)
        )


class TransportClosedError(Exception):
    pass

这也将与gremlin-pythons连接池一起使用。

如果不需要缓冲池，另一种方法是将缓冲池大小设置为1并实现某种形式的保持活动状态，如此处所述。 TINKERPOP-2352

似乎TINKERPOP-1886尚未实现gremlin-python中的Web套接字ping / keep-alive。

Gremlin Python-“服务器已断开连接-请尝试重新连接”错误

1 个答案: