Question

创建了一个包含/airflow/xcom/return.json的图片在所有子目录上使用chmod + x 由于日志显示无法找到文件或目录（尝试chmod + x）

strtpodbefore = KubernetesPodOperator(namespace='rguonew',
                                    image="mydockerimage",
                                    name="fail",
                                    image_pull_policy="Always",
                                    task_id="failing-task",
                                    get_logs= True,
                                    xcom_push=True,
                                    dag=dag
                                    )'

这是日志

[2019-03-18 20:32:07,007] {logging_mixin.py:95} INFO - [2019-03-18 
 20:32:07,007] {pod_launcher.py:166} INFO - Running command... cat 
/airflow/xcom/return.json
[2019-03-18 20:32:07,007] {logging_mixin.py:95} INFO - 
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - [2019-03-18 
20:32:07,026] {pod_launcher.py:173} INFO - cat: can't open 
 '/airflow/xcom/return.json': No such file or directory
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - 
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - [2019-03-18 
 20:32:07,026] {pod_launcher.py:166} INFO - Running command... kill -s 
 SIGINT 1
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - 
[2019-03-18 20:32:07,067] {models.py:1788} ERROR - Pod Launching 
failed: Failed to extract xcom from pod: fail-e18e3dac

因此尝试过这种方法，但它意味着它从外部而不是从图像分配xcom json

 return_value = '{"foo": "bar"\n, "buzz": 2}'
 strtpodbefore = KubernetesPodOperator(namespace='rguonew',
                                image="python:3.6.6-stretch",
                                cmds=["bash", "-cx"],
                                name="fail",
                                task_id="failing-task",
                                arguments=['echo \'{}\' > 
 /airflow/xcom/return.json'.format(return_value)],
                                get_logs= True,
                                xcom_push=True,
                                dag=dag
                                )

所以我尝试了额外的参数来尝试最终的解决方案，但仍然无法正常工作，第一个命令没有得到这样的目录返回

strtpodbefore = KubernetesPodOperator(namespace='rguonew',
                                    image="myimages",
                                    name="fail",
                                    image_pull_policy="Always",
                                    cmds=["bash", "-cx"],
                                    arguments=['echo \'{}\' > 
/airflow/xcom/return.json'.format(return_value)],
                                    task_id="failing-task",
                                    get_logs= True,
                                    xcom_push=True,
                                    dag=dag
                                    )

Answer 1

设置xcom_push = True时，将与您的执行者容器一起创建一个sidecar容器。边车容器将读取执行者的/airflow/xcom/return.json，因此实际上您需要像第二个示例中一样从执行者容器中写入内容。

请参阅此处： https://stackoverflow.com/a/53568710/10675601

在处理pod xcom时，还需要设置一些与RBAC有关的问题： Airflow k8s operator xcom - Handshake status 403 Forbidden

没有这样的文件或目录/airflow/xcom/return.json

1 个答案: