我正在使用气流，aws redshift和s3开发数据管道。我正在尝试通过docker启动气流，但是运行docker-compose up时出现错误。
错误：
Creating airflow_postgres_1 ... done
Creating airflow_webserver_1 ... done
Attaching to airflow_postgres_1, airflow_webserver_1
postgres_1 | The files belonging to this database system will be owned by user "postgres".
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale "en_US.utf8".
postgres_1 | The default database encoding has accordingly been set to "UTF8".
postgres_1 | The default text search configuration will be set to "english".
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1 | creating subdirectories ... ok
postgres_1 | selecting default max_connections ... 100
postgres_1 | selecting default shared_buffers ... 128MB
postgres_1 | selecting default timezone ... Etc/UTC
postgres_1 | selecting dynamic shared memory implementation ... posix
postgres_1 | creating configuration files ... ok
postgres_1 | running bootstrap script ... ok
postgres_1 | performing post-bootstrap initialization ... ok
postgres_1 | syncing data to disk ... ok
postgres_1 |
postgres_1 | WARNING: enabling "trust" authentication for local connections
postgres_1 | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1 | --auth-local and --auth-host, the next time you run initdb.
postgres_1 |
postgres_1 | Success. You can now start the database server using:
postgres_1 |
postgres_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1 |
postgres_1 | waiting for server to start....LOG: database system was shut down at 2020-05-20 22:59:54 UTC
postgres_1 | LOG: MultiXact member wraparound protections are now enabled
postgres_1 | LOG: database system is ready to accept connections
postgres_1 | LOG: autovacuum launcher started
postgres_1 | done
postgres_1 | server started
postgres_1 | CREATE DATABASE
postgres_1 |
postgres_1 |
postgres_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgres_1 |
postgres_1 | LOG: received fast shutdown request
postgres_1 | LOG: aborting any active transactions
postgres_1 | LOG: autovacuum launcher shutting down
postgres_1 | waiting for server to shut down....LOG: shutting down
postgres_1 | LOG: database system is shut down
webserver_1 | DB: postgresql://airflow_user:***@postgres/airflow
webserver_1 | [2020-05-20 22:59:57,188] {db.py:378} INFO - Creating tables
webserver_1 | Traceback (most recent call last):
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2345, in _wrap_pool_connect
webserver_1 | return fn()
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 304, in unique_connection
webserver_1 | return _ConnectionFairy._checkout(self)
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout
webserver_1 | fairy = _ConnectionRecord.checkout(pool)
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 495, in checkout
webserver_1 | rec = pool._do_get()
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/impl.py", line 140, in _do_get
webserver_1 | self._dec_overflow()
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit__
webserver_1 | exc_value, with_traceback=exc_tb,
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
webserver_1 | raise exception
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/impl.py", line 137, in _do_get
webserver_1 | return self._create_connection()
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
webserver_1 | return _ConnectionRecord(self)
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 440, in __init__
webserver_1 | self.__connect(first_connect_check=True)
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
webserver_1 | pool.logger.debug("Error on connect(): %s", e)
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit__
webserver_1 | exc_value, with_traceback=exc_tb,
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
webserver_1 | raise exception
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 656, in __connect
webserver_1 | connection = pool._invoke_creator(self)
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
webserver_1 | return dialect.connect(*cargs, **cparams)
webserver_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 490, in connect
webserver_1 | return self.dbapi.connect(*cargs, **cparams)
webserver_1 | File "/usr/local/lib/python3.7/site-packages/psycopg2/__init__.py", line 127, in connect
webserver_1 | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
webserver_1 | psycopg2.OperationalError: could not connect to server: Connection refused
webserver_1 | Is the server running on host "postgres" (172.24.0.2) and accepting
webserver_1 | TCP/IP connections on port 5432?
到目前为止，我已经尝试过：
airflow.cfg中的sql_alchemy_conn字符串不同（均导致与上述相同的连接被拒绝错误）
sql_alchemy_conn = postgresql://redshiftuser:redshiftpassword@host:5439/db
sql_alchemy_conn = postgresql://postgresuser:password@127.0.0.1:5432/db
sql_alchemy_conn = postgresql+psycopg2://postgresuser:password@postgres:5432/db
sql_alchemy_conn = postgresql+psycopg2://postgresuser:postgresuserpassword@postgres:5432/db
sql_alchemy_conn = postgresql+psycopg2://postgresuser:postgresspassword@localhost:5432/db
sql_alchemy_conn = postgresql://postgresuser:postgresspassword@localhost:5432/db
sql_alchemy_conn = postgresql+psycopg2://postgresuser:databasepassword@localhost:5432/db
sql_alchemy_conn = postgresql://postgresuser:databasepassword@postgres/db
在postgres.conf文件中，我进行了更改：
listen_addresses = 'localhost'
到
listen_addresses = '*'
在postgres UI中尝试过：
ALTER DATABASE airflow CONNECTION LIMIT 5;
ALTER SYSTEM SET listen_addresses = '*';
docker-compose.yaml的相关部分：
version: '3.7'
services:
postgres:
image: postgres:9.6
environment:
- POSTGRES_USER=airflow_user
- POSTGRES_PASSWORD=password
- POSTGRES_DB=airflow
logging:
options:
max-size: 10m
max-file: "3"
ports:
- "5432"
github回购链接： https://github.com/marshall7m/data_engineering_capstone/tree/master/airflow
答案 0 :(得分：2)
您从未尝试按照docker-compose.yaml的定义尝试正确的用户名，密码，数据库和主机组合！
您的主机是postgres，它与docker-compose.yaml中的服务名称相同，其他值可以从environment变量中清除
连接字符串应为以下内容：
postgresql://airflow_user:password@postgres:5439/airflow
与问题无关，但请切勿向git提交纯文本密码（即使它是私人仓库），也可以使用 Mozilla sops 来加密值或类似内容。另外，请避免使用默认密码 pwgen 或任何随机密码生成器。