我们正在尝试部署气流版本（1.9.0），该版本使用Postgres作为其数据库，并使用Redis来实现Kubernetes中Pod之间的连接。
相同的设置在暂存环境中工作正常，而在生产环境中却失败了，经过调查，我知道最近映像9.6的Postgres基本版本已更改，会不会引起任何问题？
随附的是postgress和WebServer（airflow）的日志
尝试使用不同的姿势图像但仍然相同
WebServer日志
Collecting botocore
Downloading https://files.pythonhosted.org/packages/a1/b0/7a8794d914b95ef3335a5a4ba20595b46081dbd1e29f13812eceacf091ca/botocore-1.12.215-py2.py3-none-any.whl (5.7MB)
Collecting docutils<0.16,>=0.10 (from botocore)
Downloading https://files.pythonhosted.org/packages/22/cd/a6aa959dca619918ccb55023b4cb151949c64d4d5d55b3f4ffd7eee0c6e8/docutils-0.15.2-py3-none-any.whl (547kB)
Collecting jmespath<1.0.0,>=0.7.1 (from botocore)
Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Requirement already satisfied: urllib3<1.26,>=1.20; python_version >= "3.4" in /usr/lib/python3/dist-packages (from botocore) (1.22)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1; python_version >= "2.7" in /usr/local/lib/python3.6/dist-packages (from botocore) (2.8.0)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil<3.0.0,>=2.1; python_version >= "2.7"->botocore) (1.11.0)
Installing collected packages: docutils, jmespath, botocore
Successfully installed botocore-1.12.215 docutils-0.15.2 jmespath-0.9.4
Multi-tenant details not configured in this instance - Exiting
Cluster "abc" set.
User "abc@airflow.com" set.
Context "abc" created.
Switched to context "cedp".
[2019-08-26 14:01:03,391] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2019-08-26 14:01:03,415] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 17, in <module>
from airflow.bin.cli import CLIFactory
File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 47, in <module>
from airflow import jobs, settings
File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 64, in <module>
class BaseJob(Base, LoggingMixin):
File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 96, in BaseJob
executor=executors.GetDefaultExecutor(),
File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py", line 42, in GetDefaultExecutor
DEFAULT_EXECUTOR = _get_executor(executor_name)
File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py", line 60, in _get_executor
from airflow.executors.celery_executor import CeleryExecutor
File "/usr/local/lib/python2.7/dist-packages/airflow/executors/celery_executor.py", line 18, in <module>
from celery import Celery
File "/usr/local/lib/python2.7/dist-packages/celery/local.py", line 509, in __getattr__
module = __import__(self._object_origins[name], None, None, [name])
File "/usr/local/lib/python2.7/dist-packages/celery/app/__init__.py", line 5, in <module>
from celery import _state
File "/usr/local/lib/python2.7/dist-packages/celery/_state.py", line 17, in <module>
from celery.utils.threads import LocalStack
File "/usr/local/lib/python2.7/dist-packages/celery/utils/__init__.py", line 9, in <module>
from .nodenames import worker_direct, nodename, nodesplit
File "/usr/local/lib/python2.7/dist-packages/celery/utils/nodenames.py", line 9, in <module>
from kombu.entity import Exchange, Queue
File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 9, in <module>
from .serialization import prepare_accept_content
File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 456, in <module>
for ep, args in entrypoints('kombu.serializers'): # pragma: no cover
File "/usr/local/lib/python2.7/dist-packages/kombu/utils/compat.py", line 89, in entrypoints
for ep in importlib_metadata.entry_points().get(namespace, [])
File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 456, in entry_points
ordered = sorted(eps, key=by_group)
File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 454, in <genexpr>
dist.entry_points for dist in distributions())
File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 364, in <genexpr>
cls._search_path(path, pattern)
File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 373, in _switch_path
return pathlib.Path(path)
File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 1256, in __new__
self = cls._from_parts(args, init=False)
File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 898, in _from_parts
drv, root, parts = self._parse_args(args)
File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 891, in _parse_args
return cls._flavour.parse_parts(parts)
File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 250, in parse_parts
parsed.append(intern(x))
TypeError: can't intern subclass of string
[2019-08-26 14:01:04,253] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2019-08-26 14:01:04,277] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
Traceback (most recent call last):
File "/usr/local/airflow/set_auth.py", line 16, in <module>
session.commit()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 927, in commit
self.transaction.commit()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 467, in commit
self._prepare_impl()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 447, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2209, in flush
self._flush(objects)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2329, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2293, in _flush
flush_context.execute()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
rec.execute(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
uow
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
mapper, table, insert)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 835, in _emit_insert_statements
execute(statement, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
exc_info
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 470, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "users" does not exist
LINE 1: INSERT INTO users (username, email, password) VALUES ('admin...
^
[SQL: 'INSERT INTO users (username, email, password) VALUES (%(username)s, %(email)s, %(password)s) RETURNING users.id'] [parameters: {'username': 'admin', 'password': '$2b$12$F.8CTth9cL5G9f.pd180Duz/nC8S5KwTctwf/jG1Y/QB8PZagkTa.', 'email': 'abc@airflow.com'}]
[2019-08-26 14:01:15,229] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2019-08-26 14:01:15,253] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 17, in <module>
from airflow.bin.cli import CLIFactory
File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 47, in <module>
from airflow import jobs, settings
File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 64, in <module>
class BaseJob(Base, LoggingMixin):
File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 96, in BaseJob
executor=executors.GetDefaultExecutor(),
File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py", line 42, in GetDefaultExecutor
DEFAULT_EXECUTOR = _get_executor(executor_name)
File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py", line 60, in _get_executor
from airflow.executors.celery_executor import CeleryExecutor
File "/usr/local/lib/python2.7/dist-packages/airflow/executors/celery_executor.py", line 18, in <module>
from celery import Celery
File "/usr/local/lib/python2.7/dist-packages/celery/local.py", line 509, in __getattr__
module = __import__(self._object_origins[name], None, None, [name])
File "/usr/local/lib/python2.7/dist-packages/celery/app/__init__.py", line 5, in <module>
from celery import _state
File "/usr/local/lib/python2.7/dist-packages/celery/_state.py", line 17, in <module>
from celery.utils.threads import LocalStack
File "/usr/local/lib/python2.7/dist-packages/celery/utils/__init__.py", line 9, in <module>
from .nodenames import worker_direct, nodename, nodesplit
File "/usr/local/lib/python2.7/dist-packages/celery/utils/nodenames.py", line 9, in <module>
from kombu.entity import Exchange, Queue
File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 9, in <module>
from .serialization import prepare_accept_content
File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 456, in <module>
for ep, args in entrypoints('kombu.serializers'): # pragma: no cover
File "/usr/local/lib/python2.7/dist-packages/kombu/utils/compat.py", line 89, in entrypoints
for ep in importlib_metadata.entry_points().get(namespace, [])
File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 456, in entry_points
ordered = sorted(eps, key=by_group)
File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 454, in <genexpr>
dist.entry_points for dist in distributions())
File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 364, in <genexpr>
cls._search_path(path, pattern)
File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 373, in _switch_path
return pathlib.Path(path)
File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 1256, in __new__
self = cls._from_parts(args, init=False)
File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 898, in _from_parts
drv, root, parts = self._parse_args(args)
File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 891, in _parse_args
return cls._flavour.parse_parts(parts)
File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 250, in parse_parts
parsed.append(intern(x))
TypeError: can't intern subclass of string
Postgress LOg
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default timezone ... Etc/UTC
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
waiting for server to start....LOG: database system was shut down at 2019-08-26 13:42:41 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
done
server started
CREATE DATABASE
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
LOG: received fast shutdown request
waiting for server to shut down...LOG: aborting any active transactions
.LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
LOG: database system was shut down at 2019-08-26 13:42:43 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
LOG: incomplete startup packet
LOG: incomplete startup packet
ERROR: relation "users" does not exist at character 13
STATEMENT: INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$Vmkgo0OBgjLmylPMi3yrCOhVIWhWAgrEpCCojRZw0weeP..3nneg.') RETURNING users.id
LOG: incomplete startup packet
LOG: incomplete startup packet
LOG: incomplete startup packet
LOG: incomplete startup packet
ERROR: relation "users" does not exist at character 13
STATEMENT: INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$y2DtC8uEM5coowQZP3GZsOIw/QFkqKZqvV4TcOkCSJ0wM.QbiwbA2') RETURNING users.id
LOG: incomplete startup packet
LOG: incomplete startup packet
LOG: incomplete startup packet
LOG: incomplete startup packet
ERROR: relation "users" does not exist at character 13
STATEMENT: INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$0gj.4OfVy5y.xVt2FpVny.mRfCD/1wYnAbdMA22Xj4aI54tATo4Nu') RETURNING users.id
LOG: incomplete startup packet
LOG: incomplete startup packet
ERROR: relation "users" does not exist at character 13
STATEMENT: INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$KXKdhuhdt5rmehEPxuX1He8uwE2fvgMcWoS4rg4oGzL5xWfn8Cgd6') RETURNING users.id
LOG: incomplete startup packet
LOG: incomplete startup packet
LOG: incomplete startup packet
ERROR: relation "users" does not exist at character 13
STATEMENT: INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$QVtj0DHd6uLOnlIlwbE3kezYDzP.Y8m/Ln9H9of77pEKCihOiLhnq') RETURNING users.id
LOG: incomplete startup packet
答案 0 :(得分：0)
看起来您的postgresql服务器已启动，但是气流数据库需要初始化。 （尤其是在字符13处不存在“用户”关系“用户”）
您可以尝试重新初始化气流数据库
气流initdb
答案 1 :(得分：0)
您可能需要降级您的kombu版本。 kombu==4.5.0有效。
您如何安装软件包？如果您使用的是requirements.txt文件，则它将安装kombu>4.4.0,<5.0 apache-airflow==1.9.0指定celery~=4.3
请参阅：https://github.com/celery/celery/blob/v4.3.0/requirements/default.txt
如果可能，请使用采用某种形式的锁定文件（例如pipenv或诗歌）的Python软件包管理器。