When running scheduled jobs from WhereScape RED, it's possible to configure a job to run commands when the job finishes successfully or when a task in the job fails. These commands can be set to trigger email notifications, maintenance tasks, create a log file, or other actions. 

The commands can be entered in the Job Definition screen as shown below. It is important to make sure the commands you enter reflect the operating system (Linux or Windows) where the job will run or be portable between environments, as the commands entered are simply run as a system command where the job is being executed.

Success Commands

Success commands run when a job is a marked as successful in the final stage of Job execution and the Success Command field is populated. The example used in the Job definition screen above simply outputs the values of the special tokens to a log file which also uses the tokens to form the log file name, thus creating a unique log for each run.

echo "Job $JOB_KEY$, Seq $JOB_SEQ$, Name $JOB_NAME$" >"c:\temp\$JOB_SEQ$_success_$JOB_KEY$_$JOB_NAME$.log"

Failure Commands

Failure commands run when a task marks a job as failed and the Failure Command field is populated. This command will only run once for the job even if more than one task fails in multi-threaded jobs.

If the Execute Failure Command in event of dependency failure option is set the failure command will also be run when the following conditions are met:

• The job has a Frequency set other than 'None'
• The job has at least one parent job dependency defined
• One of the parent jobs has failed

Success and Failure Command Parameters

Special Variables or "Tokens" 

• $JOB_KEY$ - replaced with the Job ID
• $JOB_SEQ$ - replaced with the Job Sequence Number
• $JOB_NAME$ - replaced with the Job Name
• $WSL_SCRIPT_<host script name in metadata>_CODE$ - Script Sourcing Tokens

RED Parameter tokens are not currently supported:

• $P<parameter_name>$ - RED Parameter tokens will be supported in a later release, instead please use the parameter read and write functions within your script if required.

Environment Variables

The following RED environment variables are set prior to running the command

• WSL_META_<variable name> - all the usual metadata environment variables available to scripts
• WSL_<JOB|TASK>_<KEY|NAME> - Job and Task environment variables
• WSL_<SEQUENCE|WORKDIR|BINDIR> - Sequence, Work Directory and Bin Directory environment variables
• WSL_SCH_RESTAPI_<URL|USER|PWD> - The Scheduler REST API URL and credentials

If the Job is executed via Azkaban Scheduler then you will also have the usual Azkaban Environment variables available, the most useful being JOB_NAME

• JOB_NAME - this equates to the name of the task in the job at the time the Success/Failure command was run, useful to determine the task in the job that has failed.  

Other Credentials

If you need to pass other credentials securely to your scripts or commands such as email server credentials then this can be achieved by utilizing one or more of the Advanced Authentication Parameters on the metadata connection. These fields will be available to the script or command as environment variables:

• WSL_META_<AUTHTKN_1|AUTHTKN_2|AUTHTKN_3|AUTHCFG>

Success and Failure Command Script Sourcing

RED can reference other scripts form the metadata at run-time, this Script Sourcing feature of RED is also available to be used in the Success and Failure Commands, this allows referencing a Host Script from the RED metadata rather than a script that already exists on the operating system.

In the following example command, you would have a python Host Script in the metadata named 'my_python_notify_script' which performs a notification based on two args, the job status and the email to notify. In the Success command you would pass an arg of 'success' and in the Failure command you can pass an arg of 'failed' so that the script can determine what the status is. The script itself will have access to the appropriate environment variables to construct a suitable email that informs of the failure or success.

python "WSL_SCRIPT_my_python_notify_script_CODE" "success" "job-admin@mydomain.com"

When a Success or Failure Command is triggered RED will do the following:

• Parse the command for RED Tokens
  • Write referenced metadata Host Scripts found in any Script Sourcing Tokens to the work directory  
  • Expand the RED Tokens in the command line, Script Sourcing Tokens are replaced with the full file path of the script
• Set up the environment variables
• Execute the expanded command
• Report if the command ran successfully or not (this has no bearing on the Job Status)

Success and Failure Command Output Protocol

Only the exit code of the Success or Failure command is considered by the Scheduler, all output streams are ignored.

• An exit code of '0' signals a successful execution of the command, this is reported in the Scheduler Job Log in RED
• A non-zero exit code signals a failure executing the command, this is also reported in the Scheduler Job Log in RED
• All output streams are ignored, therefore your script or command should perform and persist it's own logging if required.