Question

我在Centos 7中使用Python 3.7使用Airflow。

通过BashOperator运行Bash命令时，遇到以下问题：

[2019-11-13 23:20:08,238] {taskinstance.py:1058} ERROR - [Errno 2] No such file or directory: 'bash': 'bash'
Traceback (most recent call last):
  File "/home/airflow/virtualenvs/airflow_env/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 930, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/home/airflow/virtualenvs/airflow_env/lib/python3.7/site-packages/airflow/operators/bash_operator.py", line 120, in execute
    preexec_fn=pre_exec)
  File "/home/airflow/python/Python-3.7.5/Lib/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/home/airflow/python/Python-3.7.5/Lib/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'bash': 'bash'

是否需要将一个变量传递给BashOperator，以便它知道要寻找/bin/bash？在源代码中，看来BashOperator使用bash打开了一个子进程。我需要修改它以使用/bin/bash吗？

Answer 1

原来，我不得不在systemctl文件中修改PATH变量。

将:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin添加到PATH可以解决我的问题。

我的设置使用的是通过Centos 7上通过Systemctl管理的Airflow + virtualenv。

气流调度程序systemctl文件

[Unit]
Description=Airflow scheduler daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
EnvironmentFile=/etc/sysconfig/airflow
Environment=VIRTUAL_ENV=/home/airflow/virtualenvs/airflow_env
Environment=PATH=/home/airflow/virtualenvs/airflow_env/bin:$PATH:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin
User=airflow
Group=airflow
Type=simple
ExecStart=/home/airflow/virtualenvs/airflow_env/bin/airflow scheduler
Restart=always
RestartSec=5s
RuntimeDirectory=airflow
RuntimeDirectoryMode=0775

[Install]
WantedBy=multi-user.target

Airflow BashOperator找不到Bash

1 个答案: