气流服务尚未开始

时间:2019-08-28 12:47:40

标签: kubernetes webserver airflow postgres-9.6

我们正在尝试部署气流版本(1.9.0),该版本使用Postgres作为其数据库,并使用Redis来实现Kubernetes中Pod之间的连接。

相同的设置在暂存环境中工作正常,而在生产环境中却失败了,经过调查,我知道最近映像9.6的Postgres基本版本已更改,会不会引起任何问题?

随附的是postgress和WebServer(airflow)的日志

尝试使用不同的姿势图像但仍然相同

WebServer日志

Collecting botocore
  Downloading https://files.pythonhosted.org/packages/a1/b0/7a8794d914b95ef3335a5a4ba20595b46081dbd1e29f13812eceacf091ca/botocore-1.12.215-py2.py3-none-any.whl (5.7MB)
Collecting docutils<0.16,>=0.10 (from botocore)
  Downloading https://files.pythonhosted.org/packages/22/cd/a6aa959dca619918ccb55023b4cb151949c64d4d5d55b3f4ffd7eee0c6e8/docutils-0.15.2-py3-none-any.whl (547kB)
Collecting jmespath<1.0.0,>=0.7.1 (from botocore)
  Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Requirement already satisfied: urllib3<1.26,>=1.20; python_version >= "3.4" in /usr/lib/python3/dist-packages (from botocore) (1.22)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1; python_version >= "2.7" in /usr/local/lib/python3.6/dist-packages (from botocore) (2.8.0)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil<3.0.0,>=2.1; python_version >= "2.7"->botocore) (1.11.0)
Installing collected packages: docutils, jmespath, botocore
Successfully installed botocore-1.12.215 docutils-0.15.2 jmespath-0.9.4
Multi-tenant details not configured in this instance - Exiting
Cluster "abc" set.
User "abc@airflow.com" set.
Context "abc" created.
Switched to context "cedp".
[2019-08-26 14:01:03,391] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2019-08-26 14:01:03,415] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 17, in <module>
    from airflow.bin.cli import CLIFactory
  File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 47, in <module>
    from airflow import jobs, settings
  File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 64, in <module>
    class BaseJob(Base, LoggingMixin):
  File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 96, in BaseJob
    executor=executors.GetDefaultExecutor(),
  File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py", line 42, in GetDefaultExecutor
    DEFAULT_EXECUTOR = _get_executor(executor_name)
  File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py", line 60, in _get_executor
    from airflow.executors.celery_executor import CeleryExecutor
  File "/usr/local/lib/python2.7/dist-packages/airflow/executors/celery_executor.py", line 18, in <module>
    from celery import Celery
  File "/usr/local/lib/python2.7/dist-packages/celery/local.py", line 509, in __getattr__
    module = __import__(self._object_origins[name], None, None, [name])
  File "/usr/local/lib/python2.7/dist-packages/celery/app/__init__.py", line 5, in <module>
    from celery import _state
  File "/usr/local/lib/python2.7/dist-packages/celery/_state.py", line 17, in <module>
    from celery.utils.threads import LocalStack
  File "/usr/local/lib/python2.7/dist-packages/celery/utils/__init__.py", line 9, in <module>
    from .nodenames import worker_direct, nodename, nodesplit
  File "/usr/local/lib/python2.7/dist-packages/celery/utils/nodenames.py", line 9, in <module>
    from kombu.entity import Exchange, Queue
  File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 9, in <module>
    from .serialization import prepare_accept_content
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 456, in <module>
    for ep, args in entrypoints('kombu.serializers'):  # pragma: no cover
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/compat.py", line 89, in entrypoints
    for ep in importlib_metadata.entry_points().get(namespace, [])
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 456, in entry_points
    ordered = sorted(eps, key=by_group)
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 454, in <genexpr>
    dist.entry_points for dist in distributions())
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 364, in <genexpr>
    cls._search_path(path, pattern)
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 373, in _switch_path
    return pathlib.Path(path)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 1256, in __new__
    self = cls._from_parts(args, init=False)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 898, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 891, in _parse_args
    return cls._flavour.parse_parts(parts)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 250, in parse_parts
    parsed.append(intern(x))
TypeError: can't intern subclass of string
[2019-08-26 14:01:04,253] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2019-08-26 14:01:04,277] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
Traceback (most recent call last):
  File "/usr/local/airflow/set_auth.py", line 16, in <module>
    session.commit()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 927, in commit
    self.transaction.commit()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 467, in commit
    self._prepare_impl()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 447, in _prepare_impl
    self.session.flush()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2209, in flush
    self._flush(objects)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2329, in _flush
    transaction.rollback(_capture_exception=True)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2293, in _flush
    flush_context.execute()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
    rec.execute(self)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
    uow
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
    mapper, table, insert)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 835, in _emit_insert_statements
    execute(statement, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "users" does not exist
LINE 1: INSERT INTO users (username, email, password) VALUES ('admin...
                    ^
 [SQL: 'INSERT INTO users (username, email, password) VALUES (%(username)s, %(email)s, %(password)s) RETURNING users.id'] [parameters: {'username': 'admin', 'password': '$2b$12$F.8CTth9cL5G9f.pd180Duz/nC8S5KwTctwf/jG1Y/QB8PZagkTa.', 'email': 'abc@airflow.com'}]
[2019-08-26 14:01:15,229] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2019-08-26 14:01:15,253] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 17, in <module>
    from airflow.bin.cli import CLIFactory
  File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 47, in <module>
    from airflow import jobs, settings
  File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 64, in <module>
    class BaseJob(Base, LoggingMixin):
  File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 96, in BaseJob
    executor=executors.GetDefaultExecutor(),
  File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py", line 42, in GetDefaultExecutor
    DEFAULT_EXECUTOR = _get_executor(executor_name)
  File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py", line 60, in _get_executor
    from airflow.executors.celery_executor import CeleryExecutor
  File "/usr/local/lib/python2.7/dist-packages/airflow/executors/celery_executor.py", line 18, in <module>
    from celery import Celery
  File "/usr/local/lib/python2.7/dist-packages/celery/local.py", line 509, in __getattr__
    module = __import__(self._object_origins[name], None, None, [name])
  File "/usr/local/lib/python2.7/dist-packages/celery/app/__init__.py", line 5, in <module>
    from celery import _state
  File "/usr/local/lib/python2.7/dist-packages/celery/_state.py", line 17, in <module>
    from celery.utils.threads import LocalStack
  File "/usr/local/lib/python2.7/dist-packages/celery/utils/__init__.py", line 9, in <module>
    from .nodenames import worker_direct, nodename, nodesplit
  File "/usr/local/lib/python2.7/dist-packages/celery/utils/nodenames.py", line 9, in <module>
    from kombu.entity import Exchange, Queue
  File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 9, in <module>
    from .serialization import prepare_accept_content
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 456, in <module>
    for ep, args in entrypoints('kombu.serializers'):  # pragma: no cover
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/compat.py", line 89, in entrypoints
    for ep in importlib_metadata.entry_points().get(namespace, [])
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 456, in entry_points
    ordered = sorted(eps, key=by_group)
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 454, in <genexpr>
    dist.entry_points for dist in distributions())
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 364, in <genexpr>
    cls._search_path(path, pattern)
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 373, in _switch_path
    return pathlib.Path(path)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 1256, in __new__
    self = cls._from_parts(args, init=False)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 898, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 891, in _parse_args
    return cls._flavour.parse_parts(parts)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 250, in parse_parts
    parsed.append(intern(x))
TypeError: can't intern subclass of string



Postgress LOg

The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default timezone ... Etc/UTC
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok


WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

waiting for server to start....LOG:  database system was shut down at 2019-08-26 13:42:41 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
 done
server started
CREATE DATABASE


/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*

LOG:  received fast shutdown request
waiting for server to shut down...LOG:  aborting any active transactions
.LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
 done
server stopped

PostgreSQL init process complete; ready for start up.

LOG:  database system was shut down at 2019-08-26 13:42:43 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
LOG:  incomplete startup packet
LOG:  incomplete startup packet
ERROR:  relation "users" does not exist at character 13
STATEMENT:  INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$Vmkgo0OBgjLmylPMi3yrCOhVIWhWAgrEpCCojRZw0weeP..3nneg.') RETURNING users.id
LOG:  incomplete startup packet
LOG:  incomplete startup packet
LOG:  incomplete startup packet
LOG:  incomplete startup packet
ERROR:  relation "users" does not exist at character 13
STATEMENT:  INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$y2DtC8uEM5coowQZP3GZsOIw/QFkqKZqvV4TcOkCSJ0wM.QbiwbA2') RETURNING users.id
LOG:  incomplete startup packet
LOG:  incomplete startup packet
LOG:  incomplete startup packet
LOG:  incomplete startup packet
ERROR:  relation "users" does not exist at character 13
STATEMENT:  INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$0gj.4OfVy5y.xVt2FpVny.mRfCD/1wYnAbdMA22Xj4aI54tATo4Nu') RETURNING users.id
LOG:  incomplete startup packet
LOG:  incomplete startup packet
ERROR:  relation "users" does not exist at character 13
STATEMENT:  INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$KXKdhuhdt5rmehEPxuX1He8uwE2fvgMcWoS4rg4oGzL5xWfn8Cgd6') RETURNING users.id
LOG:  incomplete startup packet
LOG:  incomplete startup packet
LOG:  incomplete startup packet
ERROR:  relation "users" does not exist at character 13
STATEMENT:  INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$QVtj0DHd6uLOnlIlwbE3kezYDzP.Y8m/Ln9H9of77pEKCihOiLhnq') RETURNING users.id
LOG:  incomplete startup packet

2 个答案:

答案 0 :(得分:0)

看起来您的postgresql服务器已启动,但是气流数据库需要初始化。 (尤其是在字符13处不存在“用户”关系“用户”)

您可以尝试重新初始化气流数据库

初始化数据库

气流initdb

https://airflow.apache.org/start.html

答案 1 :(得分:0)

您可能需要降级您的kombu版本。 kombu==4.5.0有效。

您如何安装软件包?如果您使用的是requirements.txt文件,则它将安装kombu>4.4.0,<5.0 apache-airflow==1.9.0指定celery~=4.3

请参阅:https://github.com/celery/celery/blob/v4.3.0/requirements/default.txt

如果可能,请使用采用某种形式的锁定文件(例如pipenv或诗歌)的Python软件包管理器。