Page History
Current version: MT 3.0 build 250630_1433
The RED Migration Tooling allows you to migrate metadata repositories from WhereScape RED 8.6 and 9.0 to WhereScape RED 10.4+. The following sections provide information on how to install and use the RED Migration Tooling.
...
Not all object types from earlier versions or of RED are available in RED 10 so it is important to understand what will and won't be migrated, refer to the following table for more details:
...
- Finds any Load table objects without an associated Load script and attempts to generate a load script for it using the template assigned in the c4 script.
- Troubleshooting: Look at the RedCli logs in C:\ProgramData\WhereScape\Work, these will give you the most detail for any given failure. You can also see what command were sent in each batch by looking for the batch json file in the work directory of the script.
- If you do get failures then, after correcting the underlying configuration problem, subsequent reruns will only pickup failed items or items not yet generated.
Post Migration
Installing a Scheduler
You should follow the Scheduler Installation section of the RED User Guide to install a scheduler for your Destination Repository. Since the scheduler in RED 10 is java based the memory requirements will be greater than that of the RED8/9 scheduler. There are a few important things to consider when building out the scheduler infrastructure and some experiment and tuning will be required to get the optimum throughput for your workloads.
Considerations:
- Review and adjust job thread counts: The migration process capped the total thread count in jobs to 10 since this is the typical parallelism which is possible on an average machine due to memory consumption. You can begin to set this higher as your infrastructure allows.
- Review Jobs with Child Jobs: In RED 10.4.0.3 a child jobs tasks are run directly from the original job they reference, if you change a child jobs tasks then this may invalidate the parent jobs references to it which can only be fixed by re-publishing the parent job. Improved Child Job support is coming soon but until then use job nesting sparingly to avoid synchronization issues.
- If you were previously running a RED8/9 Linux scheduler then you should install an Azkaban Executor on the same machine and Linux user. If the previous workloads can not be handled on the same machine with the Azkaban Executor installed then you would scale out horizontally with more Azkaban Executor machines.
Review and Refactor Stand-alone Objects
Scripts
Review any stand-alone scripts, such as High Water Mark scripts, these may have specific code in them that calls the old metadata repository directly and/or the legacy callable routines. Additionally the script output protocol of the Azkaban Scheduler is different and the script Review any stand-alone scripts, such as High Water Mark scripts, these may have specific code in them that calls the old metadata repository directly and/or the legacy callable routines. Additionally the script output protocol of the Azkaban Scheduler is different and the script may need to be updated to conform.
Procedures and Blocks
Review stand-alone procedures and blocks, if these operated on the old metadata connection then they will have been retargeted during migration, these should be reviewed to see if they still work as expected.