Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Review and adjust job thread counts: The migration process capped the total thread count in jobs to 10 since this is the typical parallelism which is possible on an average machine due to memory consumption. You can begin to set this higher as your infrastructure allows.
  • Review Jobs with Child Jobs: In RED 10.4.0.3 a child jobs tasks are run directly from the original job they reference, if you change a child jobs tasks then this may invalidate the parent jobs references to it which can only be fixed by re-publishing the parent job. Improved Child Job support is coming soon but until then use job nesting sparingly to avoid synchronization issues. 
  • If you were previously running a RED8/9 Linux scheduler then you should install an Azkaban Executor on the same machine and Linux user. If the previous workloads can not be handled on the same machine with the Azkaban Executor installed then you would scale out horizontally with more Azkaban Executor machines.

Review and Refactor

Stand-alone Script and Procedure Objects

The term "stand-alone" in this context means script and procedure objects that are executed independently to a Table, View or Export object. These are essentially user created routines which are not generated by RED and so some refactoring post-migration may be required.

Scripts

Review any stand-alone scripts, such as High Water Mark scripts, these may have specific code in them that calls the old metadata repository directly and/or the legacy callable routines. Additionally the script output protocol of the Azkaban Scheduler is different and the script may need to be updated to conform.  

...

During refactoring consider adopting the new feature that allows scripts to be associated to Database/ODBC and Extensible Sources instead of Windows or Linux connections thus allowing secure access to that connection's credentials and settings.

SQL Blocks

SQL Blocks may have specific code in them that calls the old metadata repository directly and/or the legacy callable routines. Additionally if the associated connection was the old metadata connection then post migration the associated connection will still be that of the old metadata and so may not make sense anymore.

Procedures and Functions

Review stand-alone procedures and blocks, if these operated on the old metadata connection then they will have been retargeted during migrationcontinue to operate on the old metadata connection, these should be reviewed to see if they still work as expected and refactored to operate on the PostgreSQL meta instead if required

Review Action Script Templates

...