使用气流PythonOperator
可以模拟什么内容和位置,例如:
python_callback
引发异常,触发on_failure_callback
的调用,并且我尝试在多个地方同时模拟{python_callable}和PythonOperator.execute
,但没有成功。
代码文件如下所示:
dags / my_code.py
class CustomException(Exception): pass
def a_callable():
if OurSqlAlchemyTable.count() == 0:
raise CustomException("{} is empty".format(OurSqlAlchemyTable.name))
return True
def a_failure_callable(context):
SlackWebhookHook(
http_conn_id=slack_conn_id,
message= context['exception'].msg,
channel='#alert-channel'
).execute()
dags / a_dag.py
from my_code import a_callable, a_failure_callable
new_task = PythonOperator(
task_id='new-task', dag=dag-named-sue, conn_id='a_conn_id', timeout=30,
python_callable=a_callable,
on_failure_callback=a_failure_callable)
dags / test_a_dag.py
class TestCallback(unittest.TestCase):
def test_on_failure_callback(self):
tested_task = DagBag().get_dag('dag-named-sue').get_task('new-task')
with patch('airflow.operators.python_operator.PythonOperator.execute') as mock_execute:
with patch('dags.a_dag.a_failure_callable') as mock_callback:
mock_execute.side_effect = CustomException
tested_task.execute(context={})
# does failure of the python_callable trigger the failure callback?
mock_callback.assert_called()
# did the exception message make it to the failure callback?
failure_context = mock_callback.call_args[0]
self.assertEqual(failure_context['exception'].msg,
'OurSqlAlchemyTable is empty')O
测试确实在行CustomException
上引起self.task.execute(context={})
-但是,在测试代码本身中。我想要的是那个错误
在气流代码中引发PythonOperator
失败并调用on_failure_callback
。
我尝试了任意数量的排列,所有排列都经过测试而没有 触发,调用python_callable或找不到要修补的对象:
patch('dags.a_dag.a_callable') as mock_callable
'a_dag.a_callable'
'dags.my_code.a_callable'
'my_code.a_callable'
'airflow.models.Task.execute'
({Python3
,pytest
和mock
。)
我在想什么/做错了什么?
(更好的是,我想验证传递给SlackWebhookHook
的参数。类似:
with patch('???.SlackWebhookHook.execute') as mock_webhook:
... as above ...
kw_dict = mock_webhook.call_args[-1]
assert kw_dict['http_conn_id'] == slack_conn_id
assert kw_dict['message'] == 'OurSqlAlchemyTable is empty'
assert kw_dict['channel'] == '#alert-channel'
(但是我首先关注测试失败回调。)
谢谢。