我有一个堡垒主机可以访问的redshift实例。
我正在尝试触发一个循环CI作业,该作业会更改要重命名的db模式以包含git release标签版本。我们希望在创建针对master的git发行版时触发该事件。
CI工作流程执行以下操作:
1)通过AWS进行身份验证
2)运行python脚本,该脚本创建一个安全组,将其附加到堡垒主机上,并配置入口规则,以允许将CI盒的IP列为cidr。
3)打开一个ssh隧道,以通过该堡垒将sqlalchemy命令转发到redshift主机。
4)运行另一个更改数据库架构的python脚本
前两个步骤没有任何问题。 ssh隧道似乎正在出现问题。
ssh隧道在后台运行并调用python脚本:
ssh -o StrictHostKeyChecking=no -N -L 127.0.0.1:5439:$PRODUCTION_REDSHIFT_DNS:5439 -i /home/circleci/.ssh/id_rsa_d88b40ac655a406a2c26ef75202fc144 circle_ci@$PRODUCTION_BASTION_DNS & python3 archive_schema.py
$ PROD_REDSHIFT_CONNECTION_STRING :redshift+psycopg2://$USER:$PASS@localhost:5439/$DB_NAME
我有一个简单的python脚本,该脚本打印数据库中的第一个表名并退出。我使用相同的连接字符串在堡垒中成功运行了它,但是当我在后台设置ssh隧道后运行它时,事情就变了。
import os
from sqlalchemy import create_engine, inspect
engine = create_engine(os.getenv("PROD_REDSHIFT_CONNECTION_STRING"))
inspector = inspect(engine)
for table_name in inspector.get_table_names():
print(table_name)
break
任何帮助将不胜感激。这是从圈子CI运行脚本时遇到的错误:
Traceback (most recent call last):
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 303, in unique_connection
return _ConnectionFairy._checkout(self)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 760, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 492, in checkout
rec = pool._do_get()
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 139, in _do_get
self._dec_overflow()
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 154, in reraise
raise value
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 136, in _do_get
return self._create_connection()
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 639, in __connect
connection = pool._invoke_creator(self)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 451, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/psycopg2/__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5439?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5439?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5439?
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "z.py", line 7, in <module>
inspector = inspect(engine)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/inspection.py", line 63, in inspect
ret = reg(subject)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/reflection.py", line 141, in _insp
return Inspector.from_engine(bind)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/reflection.py", line 136, in from_engine
return bind.dialect.inspector(bind)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/base.py", line 2209, in __init__
reflection.Inspector.__init__(self, conn)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/reflection.py", line 111, in __init__
bind.connect().close()
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2193, in connect
return self._connection_cls(self, **kwargs)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 103, in __init__
else engine.raw_connection()
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2293, in raw_connection
self.pool.unique_connection, _connection
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 399, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
raise value.with_traceback(tb)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 303, in unique_connection
return _ConnectionFairy._checkout(self)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 760, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 492, in checkout
rec = pool._do_get()
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 139, in _do_get
self._dec_overflow()
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 154, in reraise
raise value
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 136, in _do_get
return self._create_connection()
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 639, in __connect
connection = pool._invoke_creator(self)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 451, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/home/circleci/repo/venv/lib/python3.6/site-packages/psycopg2/__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5439?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5439?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5439?
(Background on this error at: http://sqlalche.me/e/e3q8)
Exited with code 1