Zombie and Undead tasks in Airflow

If you have encountered such a problem, please give this article a full read.

While running top command in Unix/Linux systems, sometimes you might have encountered a task of type zombie.

Before understanding zombie task in airflow, let’s understand the workflow of a task in DAG as shown in figure below.

In the above task pipeline, worker awaits for the operator to finish. If there’s a connection failure between worker and operator or worker kills off unexpectedly, while the operator is still running, it becomes a zombie process.

Related posts:

It is a very scary time right now, and fear can paralyze us from getting work done while having all this time. So here are some ways to stay productive during the day: Work out. Working out boosts…

Physical movement and spatial awareness are two defining characteristics of true VR, so as a developer, how do you ensure that your users are getting the most out of the experience? With a new…

For the first couple of months I was doing well, but after the first one hit, they kept coming until they were almost back to back. It was pretty frustrating, knowing that my mental state was weak…