Question

我正在尝试使用Vertica Database作为元数据后端设置气流。当我尝试运行气流initdb时遇到以下错误。

以下是airflow.cfg文件中使用的配置。

sql_alchemy_conn = vertica+vertica_python://username:password@host:port/dbname

收到错误消息：

Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 37, in <module>
    args.func(args)
  File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 1131, in initdb
    db.initdb(settings.RBAC)
  File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 106, in initdb
    upgradedb()
  File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 376, in upgradedb
    command.upgrade(config, 'heads')
  File "/usr/local/lib/python3.7/site-packages/alembic/command.py", line 298, in upgrade
    script.run_env()
  File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 489, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/local/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python3.7/site-packages/alembic/util/compat.py", line 173, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.7/site-packages/airflow/migrations/env.py", line 92, in <module>
    run_migrations_online()
  File "/usr/local/lib/python3.7/site-packages/airflow/migrations/env.py", line 82, in run_migrations_online
    compare_type=COMPARE_TYPE,
  File "<string>", line 8, in configure
  File "/usr/local/lib/python3.7/site-packages/alembic/runtime/environment.py", line 822, in configure
    opts=opts,
  File "/usr/local/lib/python3.7/site-packages/alembic/runtime/migration.py", line 218, in configure
    return MigrationContext(dialect, connection, opts, environment_context)
  File "/usr/local/lib/python3.7/site-packages/alembic/runtime/migration.py", line 146, in __init__
    self.impl = ddl.DefaultImpl.get_by_dialect(dialect)(
  File "/usr/local/lib/python3.7/site-packages/alembic/ddl/impl.py", line 71, in get_by_dialect
    return _impls[dialect.name]
KeyError: 'vertica'

请告知我是否支持此配置（airflow + vertica作为后端）。对此，我们将提供任何帮助或投入。

Answer 1

是的，您可以将任何数据库用作后端。

https://airflow.apache.org/docs/stable/howto/initialize-database.html

 As Airflow was built to interact with its metadata using the great
 SqlAlchemy library, you should be able to use any database backend
 supported as a SqlAlchemy backend. We recommend using MySQL or
 Postgres.

要连接Vertica，您应该安装并配置unixODBC，

您可以在下面找到其余的内容，

Connecting to Vertica database using SQLAlchemy

Vertica作为元数据后端的气流

1 个答案: