Introduction
Like the RED Client, Azkaban Executor Servers create and use an in-memory Profile at runtime for access to the required runtime credentials for the Metadata, Sources and Target connections. Since an Azkaban job can be run on any Executor on any machine we store any required credentials (passwords encrypted) and connection strings in a central location under the "redamin"."ws_scheduler_profile" table of the RED metadata repository database.
The Azkaban Executor Servers retrieve credentials and connection strings from the "redamin"."ws_scheduler_profile" at job runtime and merges any connections missing from the profile records with those found in the RED metadata. This merging process allows for Windows style authentication to work without the need to maintain the Scheduler Profile.
Scheduler Profile Record Structure
The structure of the profile records in "redamin"."ws_scheduler_profile" is like so:
| RED Connection Name (lowercased) | ODBC Connection String | User Name | WhereScape Encrypted Password |
|---|---|---|---|
| repository | dsn=$DSN$;uid=$USER$;pwd=$PASSWORD$; | redscheduler_user | VTj0Q2xapJEWpQed8DJYvBEEnRedR94NstiHJUlLt0gC |
Encryption
WhereScape provides an encryption utility as a stand-alone tool as well as being imbedded in the applicable Azkaban components. Azkaban decryption of WhereScape encrypted passwords expects the salt used for encryption to be the lowercased connection name. Azkaban has access to the Profile Password, required for decryption, via a new property in the azkaban.locla.properties file of each Azkaban server instance. The password values in both Azkaban properties files and the Azkaban Users.xml file are also stored using encryption so that they do not appear in plain text in those files.
See encryption-util.jar section for more details on using the WhereScape encryption utility.
Maintaining the Scheduler Profile
The "redamin"."ws_scheduler_profile" table should be secured from regular users of the RED client and therefore maintenance of the records stored in this table will normally be done by an administrator user or a user with the specific permissions described below:
<insert user permissions here>
To add encrypted profile rows to the scheduler profile RED provides a script with the metadata installation which is designed to be run from within the RED UI by a user with the required permissions.
For new installs, from RED 10.2.+, you will have find the wsl_scheduler_profile_maintenance host script under your Host Script's in your RED metadata repository. For upgraded repositories you will need to load this script into RED manually from <RED_Installation_Directory>\Administrator\Scripts\wsl_scheduler_profile_maintenance.ps1
Once you execute the script you will be presented with a dialog where you can enter the Profile Password (used to encrypt any passwords entered) and each connection's credentials as well as an appropriate connection string for use by your scheduler service.
Profile Password
It is important that you enter and use the same Profile Password as you provided in your Azkaban installations otherwise Azkaban will not be able to decrypt these profile records.
Once you have done entering your Scheduler Profile for each connection, click OK.
The script will then take the passwords you have entered and encrypt them using the encryption-util.jar, with lowercased connection names as the encryption salt and the Profile Password as the encryption secret.
Next the script attempts to update the "redamin"."ws_scheduler_profile" with the Profile details provided and outputs the result and the insert statement to the results pane of RED.
This script will need to be run every time you add a new connection to RED, so that the Scheduler Profile is kept in-sync. The subsequent times you run this script the UI will be prepopulated with any existing records in the "redamin"."ws_scheduler_profile" so that you do not have to re-enter them but allows updating any records as required.