Page History

...

This job has to be run for migrating to RED10. Depending on repository size and performance this job would typically finish within 10 to 30minutes. If there are any failures in Job 2, you should view the failure reason and restart the job at the point of failure from the Azkaban Scheduler Dashboard directly, by rerunning the failed execution.

...

  Note: The sample generic action processing scripts provided in the Migration Tooling are not target specific and may need to be tweaked to work in some environments. After migration these scripts should be tested and adjusted as required. In some cases the Target EP may provide a target specific generic action processing script which you can deploy instead.

...

b8_apply_legacy_obj_subtypes

• OPTIONAL: Creates custom Object Types for legacy object subtypes which don't exist in RED 10, then finds and updates any of those object types to the new custom types.

...

c1_set_storage_templates

• Associates objects with the default storage templates of the target connection they are associated with.
• This script will not overwrite existing storage template associations for an object, it only adds missing templates.
• Included in Job '4_Set_Storage_Templates'
• If you find you were missing some default templates on one of your Destination repository Target connections, you can rerun this script to have them applied to the associated objects.

c2_generate_windows_action_scripts and c3_generate_linux_action_scripts

Depending on your Destination Repository's Scheduling platforms you can run either or both of these scripts. It is best to only run it for the platform you require first since this process can take a long time, you can always come back and run these scripts again at a later date.

Image Added

• Each script will find any object which does not already have an assigned Action Processing Script and generate the script using the associated target connection's default templates.
• Runs in batches: due to the potentially large number of objects to process, the RedCli commands are run in batches and each batch is assessed for failures before continuing with the next batch.
• When running from the UI you can monitor the progress by clicking on the command window to view the progress and time remaining (shown above).
• When run through the scheduler the same progress messages are sent to audit log and can be viewed by refreshing the audit log in the RED Scheduler tab.
• It is common to get failures in these scripts the first few times you run them, especially with large repositories with many target connections, this is usually due to missing default templates on a target connection. 
• Troubleshooting: Look at the RedCli logs in C:\ProgramData\WhereScape\Work, these will give you the most detail for any given failure. You can also see what command were sent in  each batch by looking for the batch json file in the work directory of the script.
• If you do get failures then, after correcting the underlying configuration problem, subsequent reruns will only pickup failed items or items not yet generated.

c4_set_load_templates

• Sets the load template for each Load table which has no existing load script or load template associated.
• These defaults are taken from the Load table's associated Source connection's default load script template.
• It is important to go through your source connections and make sure they have suitable load script templates assigned prior to running this script. If you have SSIS or DBLink loads these will require specific templates set.

c5_generate_load_scripts

Similar to the c2 and c3 scripts, this script runs RedCli commands in batches and progress can be viewed in the cmd window or scheduler audit trail. Some failures can be expected for the first few runs until all the configurations have been resolved.

• Finds any Load table objects without an associated Load script and attempts to generate a load script for it using the template assigned in the c4 script. 
• Troubleshooting: Look at the RedCli logs in C:\ProgramData\WhereScape\Work, these will give you the most detail for any given failure. You can also see what command were sent in  each batch by looking for the batch json file in the work directory of the script.
• If you do get failures then, after correcting the underlying configuration problem, subsequent reruns will only pickup failed items or items not yet generated.

Script Name	Purpose	Notes	Job
1_prepare_migration	Sets up required parameters for the tooling.	Auto-run on startup	n/a
2_target_repo_creation	Creates the RED metadata in the Destination PostgreSQL database.	Auto-run on startup	n/a
RUN 1_, and 2_ jobs before the following 'b#_' scripts		These two jobs should be successful before continuing 'b#_' scripts or later jobs.	1_Source_Reports 2_Migrate_Current_Objects
b1_upgrade_obj_subtypes	Updates the migrated objects subtype keys to be compatible with RED 10.	Run after Migration Job '2_Migrate_Current_Objects' completes.	3_Prepare_Target_Repository
b2_job_metadata_updates	Updates schedules and job states to be compatible with RED 10.		3_Prepare_Target_Repository
b3_storage_metadata_updates	Creates a new Metadata connection and a Target Location to represent legacy 'local' storage objects then associates those objects to the new target.		3_Prepare_Target_Repository
b4_reset_identity_sequences	Source metadata keys have been migrated to PostgreSQL as is, so the sequences in PostgreSQL need updating to reflect the migrated data. This script preforms those updates.		3_Prepare_Target_Repository
b5_target_ep_installation	Uses RedCli to install the RED 10 Target Enablement Pack from the location specified during the initial configuration step to the Destination metadata.	This script requires elevated privilege's since RedCli installs WS modules to the ProgramData directory. If the script fails, then execute it directly from RED after running RED as Administrator. Additionally the RED 10 Target EP must be on a path accessible to the Scheduler Service User if running via the Job. Unpacking the Enablement Pack locally is best.	3_Prepare_Target_Repository
b6_import_sch_integration_scripts	The RED 10 scheduler integration scripts are removed when migrating the script tables, therefore we need to import them  again.		3_Prepare_Target_Repository
b7_apply_legacy_obj_subtypes	OPTIONAL: Creates custom Object Types for legacy object subtypes which don't exist in RED 10, then updates any of those object types to the new custom types.		3_Prepare_Target_Repository
Log in to the migrated Destination repository to allow the Target Enablement pack to complete it's configurations.		Before running c#_ scripts, log in to the Destination repository and allow the RED 10 Enablement Pack post installation script to complete.
c1_set_storage_templates	Associates objects with the default storage templates of the target connection they are associated with. This update will not overwrite existing associations and only adds missing templates.		4_Set_Storage_Templates
c2_generate_windows_action_scripts	RUN EITHER c2 OR c3 OR BOTH: Generates Windows Action Scripts for all objects via RedCli Batch.	long running script, consider running via the scheduled job. You can run either c2 or c3 or both depending on your Scheduler environments.	5_Generate_Windows_Action_Scripts
c3_generate_linux_action_scripts	RUN EITHER c2 OR c3 OR BOTH: Generates Linux Action Scripts for all objects via RedCli Batch.	Long running script, consider running via the scheduled job. You can run either c2 or c3 or both depending on your Scheduler environments.	6_Generate_Linux_Action_Scripts
c4_generate_routines	OPTIONAL: Generates Load/Update routines for all objects.	This is not recommended since it will regenerate all routines using the newer RED 10 templates, instead you should generate routines via the RED UI while logged in to the Destination metadata and only do so for objects that have no associated scripts or procedures.	7_Generate_Routines

...