WhereScape is pleased to announce that WhereScape RED 10.4.0.0 is now available.
This release of RED is compatible with WhereScape 3D 9.0 versions and up.
Note
Go to https://www1.wherescape.com to see available product versions and release notes for older versions.
Log into the WhereScape Support Portal to access downloads.
Access the User Guide here.
New Features and Improvements
WhereScape presents RED 10.4 a new version of RED. The features implemented in this version are mainly focused on enabling 3rd party integrations with external tools like GIT, but also enhances the UI and enables the ability to add scripted workflows based on the current context in the RED UI.
Workflow Scripts
Workflow scripts, when defined, will appear in the context menu throughout the RED UI. When clicking on them in any context menu, the underlying script will be executed and provided with the complete object list contained by the context. This allows for integration scripts to be written that can perform actions on lists of objects, such as: creating deployment applications, checking items in and out of GIT, running external tests on the objects selected, exporting data to catalog tools, backing up scripts to disk and more!
The following changes have been made to support this new feature:
- A new Host Script Language category has been added for Workflow scripts:
- Host Script Objects assigned to a Workflow script type will appear in most context menus in RED and when selected will execute the script and provide a file containing the objects contained by the context:
- Host Script Languages can now have specific icons assigned (via Host Script Language maintenance):
Improvements for 3rd Party Integrations
- CheckIn/CheckOut feature will now trigger execution of a script, if present, to support GIT integrations. Add script names of wsl_workflow_checkin and wsl_workflow_checkout to enable this feature.
- Added new endpoints to the RED metadata service, objectsExportByName and jobsExportByName, that export metadata in a GIT friendly format.
- Deployment Application Creation via command line now accepts a specific set of objects to deploy based on the JSON structure that RED Workflow scripts produce.
- RedCli commands on script, procedure and templates now support an 'export-all' option, useful for GIT integrations. Additionally single export allows specifying a directory for auto creation of the filename and extension for the object.
3D Integration - New Merge Settings for Source Mappings
When deploying Objects with multiple Source Mappings via 3D Export to RED you can now control the merge behavior when updating existing Objects Source Mapping Sets. There is now a new option with three settings when deploying from 3D to RED.
Option: ‘Existing Source Mapping Sets will be’
- Replaced: [Default] Replaces the entire Source Mapping Set (Source Mappings can be removed in this mode)
- Merged using Application: Merges the Sets, resolving conflicts by deploying Source Mappings from this Application - Add and Update.
- Merged using Repository: Merges the Sets, resolving conflicts by retaining Source Mappings in the Repository - Add only.
This feature will be particularly useful to organizations with multiple development teams or projects affecting the same objects in RED, allowing them to deploy only the new Source Mappings that they have added and avoid deploying out-of-date changes to existing Source Mappings.
Other Improvements and Fixes
In addition to the major improvements above please see the following Detailed Changes section for the full list of customer fixes and improvements in this release.
Detailed Changes in RED 10.4.0.0
The following changes have been implemented since 10.3.0.1
Improvements
| JIRA | Release Note |
| RED-11947 | A new script category 'Workflow' has been introduced, these script types will show up in context menus and when executed provide the containing context to the script in a JSON format. |
| RED-11951 | Host Script Object names can now contain spaces and hyphens, this supports human readable display of Workflow scripts in context menus. |
| RED-11950 | Host scripts can now have individual icons set per host script type, particularly useful for Workflow scripts. New icons have been added to the installer for working with this feature. |
| RED-11957 | CheckIn/CheckOut feature will now trigger execution of a script, if present, to support GIT integrations. Add script names of wsl_workflow_checkin and wsl_workflow_checkout to enable this feature. |
| RED-11953 | Added new endpoints to the RED metadata service, objectsExportByName and jobsExportByName, that export metadata in a GIT friendly format. |
| RED-11959 | Deployment Application Creation via command line now accepts a specific set of objects to deploy based on a JSON structure that RED Workflow scripts produce. |
| RED-11955 | RedCli commands on script, procedure and templates now support an 'export-all' option, useful for GIT integrations. Additionally single export allows specifying a directory for auto creation of the filename and extension for the object. |
| RED-11759 | Updated the built in Java JRE version to openjdk 11.0.25 |
| RED-11556 | Added merge mode to application deployment for Source Mappings when deploying from WhereScape 3D |
| RED-11942 | Add Scheduler API environment variables to all script executions to support actions like releasing another job via scripts. |
| RED-12036 | Azkaban to RED metadata resilience feature now logs the failure reason during reconnection scenarios. |
| RED-11958 | RED UI now accepts the Scheduler Script Output Protocol as well as the traditional script output protocol. |
Fixes
JIRA | Release Note |
RED-11641 | Fixed Azkaban Unsupported Object Type error when scripts have [TABLEOWNER].TABLE_NAME references to Retro Objects. |
RED-12029 | Fixed issue where Windows and Linux loads were going through legacy load methods through the UI. |
RED-12028 | Prevent legacy GetKey function popups for dimensional models. |
RED-12054 | Teradata TPT Routine Load Table setting now exposed via templates. |
RED-12043 | Fixed issue where bulk regenerate load scripts fails for Named Targets when load type is not Script Based Load. |
RED-11932 | Fixed Azkaban NullPointerException when running loads from Windows or Linux source connections. |
RED-11974 | Fixed issue where interactive processing via scripts could fail with "<work_dir> contains an incorrect path" in some repositories. |
RED-11887 | Fixed issue with remapping Filegroups during application deployment with RedCli |
RED-11994 | RedDeploy and RedCli now transfer relationships metadata to to RED from 3D exports. |
RED-12004 | Table properties Update/Custom Type drop-downs now default to the correct type, script or procedure, based on the assigned default template type. |
RED-11935 | Fixed RedSetupWizard Target Connection page auto-scrolling issues. |
RED-12015 | Fixed UI issues when converting existing Facts to Partitioned Facts in RED 10.3 |
RED-12016 | Fixed issue where Optional Create Clause was not persisted for Load table objects. |
RED-12047 | Vulnerability fix in Azkaban for CVE-2024-53677 |
Important Messages When Upgrading
If there are important updates to the Azkaban metadata these will be indicated in the important upgrade messages that follow. Refer to the Upgrading Azkaban section of the RED User Guide for more details.
Important Messages when upgrading to 10.4.0.0
Note
Upgrades are cumulative; therefore you should read all upgrade messages for each version between your current RED version and this version of RED.
Upgrading RED
Version 10.4.0.0
- There are no important messages for upgrading RED to this release, but please consider the cumulative upgrade messages below when upgrading.
Version 10.3.0.0
- There are no important messages for upgrading RED to this release, but please consider the cumulative upgrade messages below when upgrading.
Version 10.2.0.0
10.2 has potentially breaking changes for upgraded repositories; therefore, upgrades should be tested thoroughly before upgrading production environments:
- All ODBC, Database, Extensible Source Connection and Scheduler Configuration credentials have moved to the RED Profile, therefore each user of RED should generate their own local profile file (see the user guide for details) to continue using RED.
- For new installs, from RED 10.2.+, you will find the wsl_scheduler_profile_maintenance host script under your Host Script's in your RED metadata repository (see the user guide for details). For upgraded repositories you will need to load this script into RED manually from <RED_Installation_Directory>\Administrator\Scripts\wsl_scheduler_profile_maintenance.ps1
- All Azkaban components require upgrading and the encrypted Scheduler profile records added to redadmin.ws_scheduler_profile (see the user guide for details) in order for jobs to run.
- Post upgrade it is encouraged to remove all credentials from the metadata since these should now reside in the in-memory session level credentials only. Run the following update on the metadata to achieve this: Remove credentials after upgrade
-- Remove credentials from Database, ODBC and Extensible Source connections UPDATE red.ws_dbc_connect SET dc_extract_userid='', dc_extract_pwd='', dc_admin_userid='', dc_admin_pwd='' WHERE dc_type IN ('D','O','Z'); -- Remove credentials from Scheduler Configuration UPDATE red.ws_wrk_external_scheduler SET wes_dashboard_userid='', wes_dashboard_pwd='', wes_restapi_userid='', wes_restapi_pwd='';
Version 10.1.0.0
10.1 has potentially breaking changes for upgraded repositories; therefore, upgrades should be tested thoroughly before upgrading production environments. The potential issues after the upgrade are the following:
- Default Routine Template settings might be lost. For object types other than Load and Export these settings have been moved from target connections to the Object Subtypes. During the metadata upgrade process, the first target-enabled connection found is used to populate the new settings under Object Subtypes. If you have multiple target-enabled connections in your repository, the optimal connection may not be used for the settings transfer. Ensure to check the Object Subtypes maintenance wizard after an upgrade to this version and set any missing/incorrect default routine templates as required. Note: this configuration can be exported from test to production once resolved in test.
- [TABLEOWNER] an $OBJECT$ token expansion no longer expands empty schemas to an additional ‘.’ in table names. This means from this release onwards you must specify the actual schema name in target location settings when schema is required on your platform. Ensure to check your target location settings for target connections before upgrading and populate the schema field, if required.
Upgrading Azkaban
Upgrading Azkaban is currently a manual process, please refer to the RED User guide for detailed upgrade instructions.
Version 10.4.0.0
- RED 10.4 improves logging from metadata connection resilience so that the actual reason given by the driver is logged when reconnecting. Additonally improvements have been made to the RED Job Plugin. The Azkaban Web Server and each Azkaban Executor will need to be upgraded to take advantage of these features.
Version 10.3.0.0
- RED 10.3 improves the metadata connection resilience in both the Azkaban Web Server component and RED Job Plugin. Each of the Azkaban metadata, Azkaban Web Server and Azkaban Executors will need to be upgraded to take advantage of these features.
Version 10.2.0.0
- In RED 10.2 Advanced Connect is always enabled and therefore there are new azkaban.local.properties fields (and response.xml tags) required that support this feature. Any automated installation scripts will need to take into account these new fields. Each of the Azkaban metadata, Azkaban Web Server and Azkaban Executors will need to be upgraded to work with this release.
Azkaban Change history
Change history for Azkaban components requiring upgrades (changes are cumulative):
- 10.4.0.0: Connection resilience logging improvements. Bug fixes for RED Job Plugin. Component upgrades.
- 10.3.0.0: Azkaban/RED metadata connection resilience improvements. Component upgrades.
- 10.2.0.0: New required fields for azkaban.local.properties and installer resposne.xml for Advanced Connect. Stand-alone script execution enhancements.
- 10.1.0.1: Log4j components upgraded to log4j2 to resolve logging issues.
- 10.1.0.0: Improvements when canceling running jobs. Component upgrades.
- 10.0.1.0: Added procedure support and long host names fix.
- 10.0.0.0: First release.
Upgrading Scheduler Integration Scripts
The default scheduler integration scripts are often improved, occasionally these improvements are required to work with a new release. Please refer to the change history below and refer to the RED user guide for detailed upgrade instructions.
Integration Scripts Change History
Change history for integration scripts requiring manual upgrade (changes are cumulative), upgrades are optional unless marked as required:
- 10.4.0.0: *** TBC ***
- 10.3.0.0: No changes
- 10.2.0.0:
- wsl_scheduler_publish - Required upgrade - Advanced Connect feature added.
- wsl_scheduler_lookup - Added version header and removed unused parameters.
- wsl_scheduler_dashboard - Added version header and removed unused parameters.
- wsl_scheduler_profile_maintenance - New script for maintaining the scheduler profile.
- 10.0.0.3: wsl_scheduler_publish - Support for republishing when existing job not found. Improved logging.
- 10.0.0.2: wsl_scheduler_publish - Added support for disabled tasks.
- 10.0.0.0: First release.