Killing a Job from Azkaban

If you need to abort a running Job then the best approach is to 'Kill' the Flow in Azkaban, which in turn will update the job status in RED thus keeping the two systems in sync. 

Navigate to the running Flow Execution and click the 'Kill' button to stop the current execution and synchronize  the job status in RED.

Cleaning up after killing jobs in Azkaban

When you Kill a Flow in Azkaban, the underlying threads of any running tasks at the time of the kill may still leave an operation in progress such as the execution of a data flow on a table in your data warehouse.

To cleanup running processes after a Kill operation one or more of these steps should be performed:

• Use task manager or process monitoring to find and kill any running Powershell and/or Python scripts related to the killed job execution.
• Abort long running data movements in the target platform using target specific tools.

RED also allows a way to automate cleanup steps if required.

Automating cleanup after killing a job

When killing a Flow (job) in Azkaban, synchronization with the job in RED will be triggered and this process will also look for the presence of a cleanup script for the kill operation.

Depending on the execution platform (Windows or Linux) the following Host Script names will be looked up in the RED metadata and the script executed if found in the case of kill operations: 

• wsl_scheduler_job_kill_linux

• wsl_scheduler_job_kill_win

Aborting a Job from RED

Aborting a job from the RED UI is possible from RED 10.5+ but this action does not synchronize the related Azkaban execution so should really only be used when your Azkaban execution has finished but RED UI is showing it as still running. 

A job can be aborted by right-clicking the job in the RED Scheduler window and choosing Abort Job from the context menu.

Once in this state, a job cannot be restarted. The job now only exists as a log of what occurred and is no longer regarded as a job.