Psycopg2访问远程主机上的PostgreSQL数据库,无需手动打开ssh隧道

时间:2014-02-26 16:01:01

标签: python database postgresql ssh psycopg2

我的标准程序PostgreSQL用于访问远程服务器上的数据库是打开的,首先创建一个ssh隧道:

ssh username1@remote.somewhere.com -L 5432:localhost:5432 -p 222

然后在python中从另一个shell运行我的查询:

conn = psycopg2.connect("host=localhost" + " dbname=" +
                         conf.dbname + " user=" + conf.user + 
                         " password=" + conf.password)

cur = conn.cursor()

cur.execute(query)

创建隧道后,这段python代码可以很好地工作。但是,我希望psycopg2已经打开SSH隧道或“以某种方式”到达远程数据库而无需在我的localhost上重定向它。

是否可以使用psycopg2执行此操作?

否则可以在我的python代码中打开ssh隧道?

如果我使用:

os.system("ssh username1@remote.somewhere.com -L 5432:localhost:5432 -p 222")

shell将被重定向到远程主机,阻止主线程的执行。

6 个答案:

答案 0 :(得分:7)

你也可以使用sshtunnel,短而甜:

from sshtunnel.sshtunnel import SSHTunnelForwarder
PORT=5432
with SSHTunnelForwarder((REMOTE_HOST, REMOTE_SSH_PORT),
         ssh_username=REMOTE_USERNAME,
         ssh_password=REMOTE_PASSWORD,
         remote_bind_address=('localhost', PORT),
         local_bind_address=('localhost', PORT)):
    conn = psycopg2.connect(...)

答案 1 :(得分:1)

通过os.system在单独的线程/进程中调用您的ssh。您还可以将-N与ssh一起使用以避免打开远程shell。

答案 2 :(得分:0)

Clodoaldo Neto的代码完美地为我工作,但要注意它之后不会清理这个过程。

Luca Fiaschi所展示的方法也适用于我。我为python3和更新的psutil模块更新了一下。更改只是process.username和process.cmdline现在是函数,迭代器是process_iter()而不是get_process_list()。

这是一个稍微修改过的代码示例,Luca Fiaschi发布了与python3一起使用的代码(需要psutil模块)。我希望它至少大部分是正确的!

#!/usr/bin/env python3

import psutil
import psycopg2
import subprocess
import time
import os

# Tunnel Config
SSH_HOST = "111.222.333.444"
SSH_USER = "user"
SSH_KEYFILE = "key.pem"
SSH_FOREIGN_PORT = 5432   # Port that postgres is running on the foreign server
SSH_INTERNAL_PORT = 5432  # Port we open locally that is forwarded to
                          # FOREIGN_PORT on the server.

# Postgres Config
DB_HOST = "127.0.0.1"
DB_PORT = SSH_INTERNAL_PORT
DB_PASSWORD = "password"
DB_DATABASE = "postgres"
DB_USER = "user"

class SSHTunnel(object):
    """
    A context manager implementation of an ssh tunnel opened from python

    """
    def __init__(self, tunnel_command):
        assert "-fN" in tunnel_command, "need to open the tunnel with -fN"
        self._tunnel_command = tunnel_command
        self._delay = 0.1
        self.ssh_tunnel = None

    def create_tunnel(self):
        tunnel_cmd = self._tunnel_command
        ssh_process = subprocess.Popen(tunnel_cmd, universal_newlines=True,
            shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
            stdin=subprocess.PIPE)

        # Assuming that the tunnel command has "-f" and "ExitOnForwardFailure=yes", then the
        # command will return immediately so we can check the return status with a poll().

        while True:
            p = ssh_process.poll()
            if p is not None: break
            time.sleep(self._delay)


        if p == 0:
            # Unfortunately there is no direct way to get the pid of the spawned ssh process, so we'll find it
            # by finding a matching process using psutil.

            current_username = psutil.Process(os.getpid()).username()
            ssh_processes = [proc for proc in psutil.process_iter() if proc.cmdline() == tunnel_cmd.split() and proc.username() == current_username]

            if len(ssh_processes) == 1:
                self.ssh_tunnel = ssh_processes[0]
                return ssh_processes[0]
            else:
                raise RuntimeError('multiple (or zero?) tunnel ssh processes found: ' + str(ssh_processes))
        else:
            raise RuntimeError('Error creating tunnel: ' + str(p) + ' :: ' + str(ssh_process.stdout.readlines()))

    def release(self):
        """ Get rid of the tunnel by killin the pid
        """
        if self.ssh_tunnel:
            self.ssh_tunnel.terminate()

    def __enter__(self):
        self.create_tunnel()
        return self

    def __exit__(self, type, value, traceback):
        self.release()

    def __del__(self):
        self.release()

command = "ssh -i %s %s@%s -fNL %d:localhost:%d"\
    % (SSH_KEYFILE, SSH_USER, SSH_HOST, SSH_INTERNAL_PORT, SSH_FOREIGN_PORT)

with SSHTunnel(command):
    conn = psycopg2.connect(host = DB_HOST, password = DB_PASSWORD,
                     database = DB_DATABASE, user = DB_USER, port = DB_PORT)
    curs = conn.cursor()
    sql = "select * from table"
    curs.execute(sql)
    rows = curs.fetchall()
    print(rows)

答案 3 :(得分:0)

使用 sshtunnel 包

我不熟悉 SSH 隧道,所以我在使用 mrts 的回答时遇到了一些困难。 也许这些精确度可以帮助某人。

在 psycopg2.connect() 中,host 和 port 是您刚刚通过 ssh 隧道连接远程主机创建的。

这是我的代码:

from sshtunnel import SSHTunnelForwarder

server = SSHTunnelForwarder((REMOTE_HOST, REMOTE_SSH_PORT),
         ssh_username=REMOTE_USERNAME,
         ssh_password=REMOTE_PASSWORD,
         remote_bind_address=('localhost', PORT),
         local_bind_address=('localhost', PORT))
server.start()

import psycopg2
conn = psycopg2.connect(
    database=DATABASE,
    user=USER,
    host=server.local_bind_host,
    port=server.local_bind_port,
    password=PWD)
cur = conn.cursor()
cur.execute("select * from yourtable limit 1;")
data = cur.fetchall()
print(data)

我希望这个例子更清楚。

答案 4 :(得分:-1)

目前我正在使用基于this gist的解决方案:

class SSHTunnel(object):
    """
    A context manager implementation of an ssh tunnel opened from python

    """


    def __init__(self, tunnel_command):

        assert "-fN" in tunnel_command, "need to open the tunnel with -fN"
        self._tunnel_command = tunnel_command
        self._delay = 0.1

    def create_tunnel(self):

        tunnel_cmd = self._tunnel_command
        import time, psutil, subprocess
        ssh_process = subprocess.Popen(tunnel_cmd,  universal_newlines=True,
                                                    shell=True,
                                                    stdout=subprocess.PIPE,
                                                    stderr=subprocess.STDOUT,
                                                    stdin=subprocess.PIPE)

        # Assuming that the tunnel command has "-f" and "ExitOnForwardFailure=yes", then the
        # command will return immediately so we can check the return status with a poll().

        while True:
            p = ssh_process.poll()
            if p is not None: break
            time.sleep(self._delay)


        if p == 0:
            # Unfortunately there is no direct way to get the pid of the spawned ssh process, so we'll find it
            # by finding a matching process using psutil.

            current_username = psutil.Process(os.getpid()).username
            ssh_processes = [proc for proc in psutil.get_process_list() if proc.cmdline == tunnel_cmd.split() and proc.username == current_username]

            if len(ssh_processes) == 1:
                self.ssh_tunnel = ssh_processes[0]
                return ssh_processes[0]
            else:
                raise RuntimeError, 'multiple (or zero?) tunnel ssh processes found: ' + str(ssh_processes)
        else:
            raise RuntimeError, 'Error creating tunnel: ' + str(p) + ' :: ' + str(ssh_process.stdout.readlines())


    def release(self):
        """ Get rid of the tunnel by killin the pid
        """
        self.ssh_tunnel.terminate()


    def __enter__(self):
        self.create_tunnel()
        return self


    def __exit__(self, type, value, traceback):

        self.release()


    def __del__(self):
        self.release()


def test():
    #do things that will fail if the tunnel is not opened

    print "done =========="


command = "ssh username@someserver.com -L %d:localhost:%d -p 222 -fN" % (someport, someport)

with SSHTunnel(command):
    test()

如果有人有更好的主意,请告诉我

答案 5 :(得分:-2)

from time import sleep

os.system("ssh username1@remote.somewhere.com -fNL 5432:localhost:5432 -p 222")

while True:
    try:
        conn = psycopg2.connect(
            "host=localhost dbname={0} user={1} password={2}".format(
                conf.dbname, conf.user, conf.password
            )
        )
        break
    except psycopg2.OperationalError:
        sleep(3)