Page History

...

These are the Migration Tooling Scripts, each script can be run from the RED UI or via the indicated Scheduled Job. If you choose to run these scripts manually, please follow the order carefully as listed here. 

Image RemovedImage AddedMT Scripts 

All the scripts, except for 1 and 2, can be rerun at anytime if required to address failures or if the job 2_Migrate_Current_Objects has been completely rerun. 

Auto-run at startup

1_prepare_migration

• Sets up required parameters for the tooling.

2_target_repo_creation 

• Creates the RED metadata in the Destination PostgreSQL database.

Run only after Job '2_Migrate_Current_Objects' scripts

The following 'b' scripts are all included in job 3_Prepare_Target_Repository

b1_upgrade_obj_subtypes

• Updates the migrated objects subtype keys to be compatible with RED 10. 

b2_job_metadata_updates

• Updates schedules and job states to be compatible with RED 10.

b3_storage_metadata_updates

• Renames the existing 'Repository' connection to 'Old RED Repository Connection'
• Creates a Target Location to represent legacy 'local' storage of migrated objects, named 'WSL_MIGRATED_LOCAL_STORAGE' on connection 'Old RED Repository Connection', then associates local storage objects to this target. The target name can be changed as required after the migration.
• Creates a new connection named 'Repository' for the new PostgreSQL Metadata connection.

b4_reset_identity_sequences

• Source metadata keys have been migrated to PostgreSQL as is, so the sequences in PostgreSQL need updating to reflect the migrated data. This script preforms those updates.
• Note: if you have to rerun any of the load objects within Job '2_Migrate_Current_Objects' then this script should be rerun to reflect the new data.

b5_target_ep_installation

• This step installs the Target Enablement Pack from the path given during the migration startup script, this path is stored and retrieved from the parameter: 'red10EPLocation'.
• Caution: if the path to the EP is a network path then the user running the scheduler service may not have access to it, this can fail the job. To resolve this you can run this script manually from RED or copy the EP to a local directory the Scheduler service can access then restart the job from the failed step in the Azkaban Dashboard by rerunning the failed execution.

b6_import_sch_integration_scripts

• The RED 10 scheduler integration scripts are removed when migrating the script tables, therefore we need to import them again, these are imported from the current version of RED running the Migration Tooling. 

b7_set_default_action_scripts

RED 10 requires each object which is processed via the Scheduler to have an Action Processing Script, for large migrated repositories generating an individual script for every object can take a very long time and can increase the metadata footprint substantially. Where possible it is more efficient to use a generic action script for most objects if they have simple scheduling requirements, this script determines those candidate objects and assigns a generic script where possible.

• This script is driven by the two parameters 'red10DefaultLinuxActionScriptName' and 'red10DefaultWindowsActionScriptName'.
• This script runs a set a queries to determine candidate objects for a generic action script and then uses the parameters above to assign a default generic action scrip to those objects. 
• The Migration Tooling provides two sample generic action processing scripts which are copied across to the Destination metadata and then used in these assignments.
• The sample sample generic action processing scripts are 'wsl_mt_py_action_script' for python and 'wsl_mt_ps_action_script'.*
• If you have your own generic action processing script in the target to assign you can set in in these parameters.
• To disable this script you can remove the script names from these parameters.

Image AddedMT Default Action Scripts

  Note: The sample generic action processing scripts provided in the Migration Tooling are not target specific and may need to be tweaked to work in some environments. After migration these scripts should be tested and adjusted as required. In some cases the Target EP may provide a target specific generic action processing script which you can deploy instead.

b7_apply_legacy_obj_subtypes

• OPTIONAL: Creates custom Object Types for legacy object subtypes which don't exist in RED 10, then finds and updates any of those object types to the new custom types.

c1_set_storage_templates

Script Name	Purpose	Notes	Job
1_prepare_migration	Sets up required parameters for the tooling.	Auto-run on startup	n/a
2_target_repo_creation	Creates the RED metadata in the Destination PostgreSQL database.	Auto-run on startup	n/a
RUN 1_, and 2_ jobs before the following 'b#_' scripts		These two jobs should be successful before continuing 'b#_' scripts or later jobs.	1_Source_Reports 2_Migrate_Current_Objects
b1_upgrade_obj_subtypes	Updates the migrated objects subtype keys to be compatible with RED 10.	Run after Migration Job '2_Migrate_Current_Objects' completes.	3_Prepare_Target_Repository
b2_job_metadata_updates	Updates schedules and job states to be compatible with RED 10.		3_Prepare_Target_Repository
b3_storage_metadata_updates	Creates a new Metadata connection and a Target Location to represent legacy 'local' storage objects then associates those objects to the new target.		3_Prepare_Target_Repository
b4_reset_identity_sequences	Source metadata keys have been migrated to PostgreSQL as is, so the sequences in PostgreSQL need updating to reflect the migrated data. This script preforms those updates.		3_Prepare_Target_Repository
b5_target_ep_installation	Uses RedCli to install the RED 10 Target Enablement Pack from the location specified during the initial configuration step to the Destination metadata.	This script requires elevated privilege's since RedCli installs WS modules to the ProgramData directory. If the script fails, then execute it directly from RED after running RED as Administrator. Additionally the RED 10 Target EP must be on a path accessible to the Scheduler Service User if running via the Job. Unpacking the Enablement Pack locally is best.	3_Prepare_Target_Repository
b6_import_sch_integration_scripts	The RED 10 scheduler integration scripts are removed when migrating the script tables, therefore we need to import them  again.		3_Prepare_Target_Repository
b7_apply_legacy_obj_subtypes	OPTIONAL: Creates custom Object Types for legacy object subtypes which don't exist in RED 10, then updates any of those object types to the new custom types.		3_Prepare_Target_Repository
Log in to the migrated Destination repository to allow the Target Enablement pack to complete it's configurations.		Before running c#_ scripts, log in to the Destination repository and allow the RED 10 Enablement Pack post installation script to complete.
c1_set_storage_templates	Associates objects with the default storage templates of the target connection they are associated with. This update will not overwrite existing associations and only adds missing templates.		4_Set_Storage_Templates
c2_generate_windows_action_scripts	RUN EITHER c2 OR c3 OR BOTH: Generates Windows Action Scripts for all objects via RedCli Batch.	long running script, consider running via the scheduled job. You can run either c2 or c3 or both depending on your Scheduler environments.	5_Generate_Windows_Action_Scripts
c3_generate_linux_action_scripts	RUN EITHER c2 OR c3 OR BOTH: Generates Linux Action Scripts for all objects via RedCli Batch.	Long running script, consider running via the scheduled job. You can run either c2 or c3 or both depending on your Scheduler environments.	6_Generate_Linux_Action_Scripts
c4_generate_routines	OPTIONAL: Generates Load/Update routines for all objects.	This is not recommended since it will regenerate all routines using the newer RED 10 templates, instead you should generate routines via the RED UI while logged in to the Destination metadata and only do so for objects that have no associated scripts or procedures.	7_Generate_Routines

...