Versions Compared

Old Version 7

changes.mady.by.user Henry Sage

Saved on

compared with

New Version Current

changes.mady.by.user eric revollo

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

When Upgrading from previous versions of RED, the RED metadata, Azkaban metadata or both may need upgrading. For RED the requirement to upgrade is enforced at login when there is an application version mismatch, but it is best practice to validate and upgrade the metadata after every RED application upgrade. Refer to the Upgrading WhereScape RED section of the RED User Guide for more details.

If there are important updates to the Azkaban metadata these will be indicated in the important upgrade messages that follow. Refer to the Upgrading Azkaban section of the RED User Guide for more details.

...

Upgrading RED

Version 10.7.0.0

  • There are no important messages for upgrading RED to this release, but please consider the cumulative upgrade messages below when upgrading.

Version 10.6.0.0

  • This version adds new RED Metadata indexes to improve query performance, these indexes will be applied automatically when missing during upgrades.

...

  • All ODBC, Database, Extensible Source Connection and Scheduler Configuration credentials have moved to the RED Profile, therefore each user of RED should generate their own local profile file (see the user guide for details) to continue using RED.
  • For new installs, from RED 10.2.+, you will find the wsl_scheduler_profile_maintenance host script under your Host Script's in your RED metadata repository (see the user guide for details). For upgraded repositories you will need to load this script into RED manually from <RED_Installation_Directory>\Administrator\Scripts\wsl_scheduler_profile_maintenance.ps1
  • All Azkaban components require upgrading and the encrypted Scheduler profile records added to redadmin.ws_scheduler_profile (see the user guide for details) in order for jobs to run.
  • Post upgrade it is encouraged to remove all credentials from the metadata since these should now reside in the in-memory session level credentials only. Run the following update on the metadata to achieve this: 
    Code Block
    languagesql
    titleRemove credentials after upgrade
    -- Remove credentials from Database, ODBC and Extensible Source connections
UPDATE red.ws_dbc_connect
SET dc_extract_userid='', dc_extract_pwd='', dc_admin_userid='', dc_admin_pwd=''
WHERE dc_type IN ('D','O','Z');

-- Remove credentials from Scheduler Configuration
UPDATE red.ws_wrk_external_scheduler
SET wes_dashboard_userid='', wes_dashboard_pwd='', wes_restapi_userid='', wes_restapi_pwd='';

...

Upgrading Azkaban

Upgrading Azkaban is currently on Windows is handled by the RED Scheduler Installer, however upgrading on Linux is a manual process, please refer to the RED User guide for detailed detailed upgrade instructions.

Version 10.7.0.0

  • Added data input validation checks during installationRED 10.7.0.0 resolves an issue, introduced in 10.5, where the RED Scheduler Plugin was not completely flushing the output stream leading to errors for some python based scripts. Additionally PowerShell scripts on Windows now include a BOM. The Azkaban Web Server and each Azkaban Executor will need to be upgraded to work with this release.

Version 10.6.0.2

  • RED 10.6.0.2 added support for Job Success and Failure commands. The Azkaban Web Server and each Azkaban Executor will need to be upgraded to work with this release.
    Note

    If you had already implemented Success and Failure commands via customizations to the scheduler publish script using Azkaban conditional flow logic then these publish script changes should be reverted and those jobs republished, otherwise Success and Failure command execution may be duplicated.

     

...

Change history for Azkaban components requiring upgrades (changes are cumulative):

  • 10.0.7.0: Added validation checks during installationResolved an output stream issue with RED Scheduler Plugin and added BOM to Windows PowerShell scripts.
  • 10.0.6.2: Added Job Success and Failure command support to the RED Scheduler Plugin.
  • 10.6.0.0: Added Extensible Authentication feature affecting the Profile metadata table and script execution. 
  • 10.5.0.0: Added functionality to remove interactive scheduler jobs after execution.
  • 10.4.0.2: SchedulerUtils.jar has a new method that allows running RED script objects standalone.
  • 10.4.0.1: Standalone Batch\Unix\Linux script support. Fixes for resolving references to Dim View objects.
  • 10.4.0.0: Connection resilience logging improvements. Bug fixes for RED Job Plugin. Component upgrades.
  • 10.3.0.0: Azkaban/RED metadata connection resilience improvements. Component upgrades.
  • 10.2.0.0: New required fields for azkaban.local.properties and installer resposne.xml for Advanced Connect. Stand-alone script execution enhancements.
  • 10.1.0.1: Log4j components upgraded to log4j2 to resolve logging issues.
  • 10.1.0.0: Improvements when canceling running jobs. Component upgrades.
  • 10.0.1.0: Added procedure support and long host names fix.
  • 10.0.0.0: First release.

...

The default scheduler integration scripts are often improved, occasionally these improvements are required to work with a new release. Please refer to the change history below and refer to the RED user guide for detailed upgrade instructions.

Integration Scripts Change History

...