`docker-compose up`：postgres连接被拒绝

时间：2020-05-21 20:41:54

标签： python postgresql docker docker-compose

我正在使用气流，aws redshift和s3开发数据管道。我正在尝试通过docker启动气流，但是运行docker-compose up时出现错误。

错误：

Creating airflow_postgres_1 ... done
Creating airflow_webserver_1 ... done
Attaching to airflow_postgres_1, airflow_webserver_1
postgres_1   | The files belonging to this database system will be owned by user "postgres".
postgres_1   | This user must also own the server process.
postgres_1   | 
postgres_1   | The database cluster will be initialized with locale "en_US.utf8".
postgres_1   | The default database encoding has accordingly been set to "UTF8".
postgres_1   | The default text search configuration will be set to "english".
postgres_1   | 
postgres_1   | Data page checksums are disabled.
postgres_1   | 
postgres_1   | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1   | creating subdirectories ... ok
postgres_1   | selecting default max_connections ... 100
postgres_1   | selecting default shared_buffers ... 128MB
postgres_1   | selecting default timezone ... Etc/UTC
postgres_1   | selecting dynamic shared memory implementation ... posix
postgres_1   | creating configuration files ... ok
postgres_1   | running bootstrap script ... ok
postgres_1   | performing post-bootstrap initialization ... ok
postgres_1   | syncing data to disk ... ok
postgres_1   | 
postgres_1   | WARNING: enabling "trust" authentication for local connections
postgres_1   | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1   | --auth-local and --auth-host, the next time you run initdb.
postgres_1   | 
postgres_1   | Success. You can now start the database server using:
postgres_1   | 
postgres_1   |     pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1   | 
postgres_1   | waiting for server to start....LOG:  database system was shut down at 2020-05-20 22:59:54 UTC
postgres_1   | LOG:  MultiXact member wraparound protections are now enabled
postgres_1   | LOG:  database system is ready to accept connections
postgres_1   | LOG:  autovacuum launcher started
postgres_1   |  done
postgres_1   | server started
postgres_1   | CREATE DATABASE
postgres_1   | 
postgres_1   | 
postgres_1   | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgres_1   | 
postgres_1   | LOG:  received fast shutdown request
postgres_1   | LOG:  aborting any active transactions
postgres_1   | LOG:  autovacuum launcher shutting down
postgres_1   | waiting for server to shut down....LOG:  shutting down
postgres_1   | LOG:  database system is shut down
webserver_1  | DB: postgresql://airflow_user:***@postgres/airflow
webserver_1  | [2020-05-20 22:59:57,188] {db.py:378} INFO - Creating tables
webserver_1  | Traceback (most recent call last):
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2345, in _wrap_pool_connect
webserver_1  |     return fn()
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 304, in unique_connection
webserver_1  |     return _ConnectionFairy._checkout(self)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout
webserver_1  |     fairy = _ConnectionRecord.checkout(pool)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 495, in checkout
webserver_1  |     rec = pool._do_get()
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/impl.py", line 140, in _do_get
webserver_1  |     self._dec_overflow()
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit__
webserver_1  |     exc_value, with_traceback=exc_tb,
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
webserver_1  |     raise exception
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/impl.py", line 137, in _do_get
webserver_1  |     return self._create_connection()
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
webserver_1  |     return _ConnectionRecord(self)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 440, in __init__
webserver_1  |     self.__connect(first_connect_check=True)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
webserver_1  |     pool.logger.debug("Error on connect(): %s", e)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit__
webserver_1  |     exc_value, with_traceback=exc_tb,
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
webserver_1  |     raise exception
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 656, in __connect
webserver_1  |     connection = pool._invoke_creator(self)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
webserver_1  |     return dialect.connect(*cargs, **cparams)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 490, in connect
webserver_1  |     return self.dbapi.connect(*cargs, **cparams)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/psycopg2/__init__.py", line 127, in connect
webserver_1  |     conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
webserver_1  | psycopg2.OperationalError: could not connect to server: Connection refused
webserver_1  |  Is the server running on host "postgres" (172.24.0.2) and accepting
webserver_1  |  TCP/IP connections on port 5432?

到目前为止，我已经尝试过：

airflow.cfg中的sql_alchemy_conn字符串不同（均导致与上述相同的连接被拒绝错误）

sql_alchemy_conn = postgresql://redshiftuser:redshiftpassword@host:5439/db
sql_alchemy_conn = postgresql://postgresuser:password@127.0.0.1:5432/db
sql_alchemy_conn = postgresql+psycopg2://postgresuser:password@postgres:5432/db
sql_alchemy_conn = postgresql+psycopg2://postgresuser:postgresuserpassword@postgres:5432/db
sql_alchemy_conn = postgresql+psycopg2://postgresuser:postgresspassword@localhost:5432/db
sql_alchemy_conn = postgresql://postgresuser:postgresspassword@localhost:5432/db
sql_alchemy_conn = postgresql+psycopg2://postgresuser:databasepassword@localhost:5432/db
sql_alchemy_conn = postgresql://postgresuser:databasepassword@postgres/db

在postgres.conf文件中，我进行了更改：

listen_addresses = 'localhost'

到

listen_addresses = '*'

在postgres UI中尝试过：

ALTER DATABASE airflow CONNECTION LIMIT 5; ALTER SYSTEM SET listen_addresses = '*';

docker-compose.yaml的相关部分：

version: '3.7'
services: 
  postgres: 
    image: postgres:9.6 
    environment: 
        - POSTGRES_USER=airflow_user
        - POSTGRES_PASSWORD=password
        - POSTGRES_DB=airflow
     logging:
       options:
         max-size: 10m
         max-file: "3"
     ports:
        - "5432"

github回购链接： https://github.com/marshall7m/data_engineering_capstone/tree/master/airflow

1 个答案:

答案 0 :(得分：2)

您从未尝试按照docker-compose.yaml的定义尝试正确的用户名，密码，数据库和主机组合！

您的主机是postgres，它与docker-compose.yaml中的服务名称相同，其他值可以从environment变量中清除

连接字符串应为以下内容：

postgresql://airflow_user:password@postgres:5439/airflow

与问题无关，但请切勿向git提交纯文本密码（即使它是私人仓库），也可以使用 Mozilla sops 来加密值或类似内容。另外，请避免使用默认密码 pwgen 或任何随机密码生成器。

相关问题
最新问题