While running a DAG which runs a jar using a docker image,
xcom_push=True is given which creates another container along with the docker image in a sing
This happened because the result of the task execution is not being pushed to the xcom in the expected path required by the KubernetesPodOperator plugin. Take a look at the following unit test from the Airflow repository to check how it should be implemented (source code snippet included below for your convenience, followed by the link to the repository):
def test_xcom_push(self):
return_value = '{"foo": "bar"\n, "buzz": 2}'
k = KubernetesPodOperator(
namespace='default',
image="ubuntu:16.04",
cmds=["bash", "-cx"],
arguments=['echo \'{}\' > /airflow/xcom/return.json'.format(return_value)],
labels={"foo": "bar"},
name="test",
task_id="task",
xcom_push=True
)
self.assertEqual(k.execute(None), json.loads(return_value))
https://github.com/apache/incubator-airflow/blob/36f3bfb0619cc78698280f6ec3bc985f84e58343/tests/contrib/minikube/test_kubernetes_pod_operator.py#L321
edit: it is worth mentioning that the result pushed to the xcom must be a json.