Failed to extract xcom from airflow pod - Kubernetes Pod Operator

前端 未结 3 455
感动是毒
感动是毒 2021-01-13 02:45

While running a DAG which runs a jar using a docker image,
xcom_push=True is given which creates another container along with the docker image in a sing

3条回答
  •  野的像风
    2021-01-13 03:13

    This happened because the result of the task execution is not being pushed to the xcom in the expected path required by the KubernetesPodOperator plugin. Take a look at the following unit test from the Airflow repository to check how it should be implemented (source code snippet included below for your convenience, followed by the link to the repository):

        def test_xcom_push(self):
            return_value = '{"foo": "bar"\n, "buzz": 2}'
            k = KubernetesPodOperator(
                namespace='default',
                image="ubuntu:16.04",
                cmds=["bash", "-cx"],
                arguments=['echo \'{}\' > /airflow/xcom/return.json'.format(return_value)],
                labels={"foo": "bar"},
                name="test",
                task_id="task",
                xcom_push=True
            )
            self.assertEqual(k.execute(None), json.loads(return_value))
    

    https://github.com/apache/incubator-airflow/blob/36f3bfb0619cc78698280f6ec3bc985f84e58343/tests/contrib/minikube/test_kubernetes_pod_operator.py#L321

    edit: it is worth mentioning that the result pushed to the xcom must be a json.

提交回复
热议问题