Page History

Versions Compared

Old Version 2

changes.mady.by.user Morgan Mackeson-Ellis

Saved on Aug 08, 2024

compared with

New Version Current

changes.mady.by.user eric revollo

Saved on Sep 19, 2024

Key

This line was added.
This line was removed.
Formatting was changed.

...

Currently, there is no interactive install wizard for WhereScape Azkaban on Linux but the azkaban-installer.jar file in the RED install directory under <install_dir>\Azkaban\ can be transferred to Linux and run directly using Java. This process requires a pre-prepared response file to be supplied with the parameters for the installation.

...

First, set the environment variables for your Azkaban metadata user and pwd:
AZ_DB_USER=myuser
AZ_DB_PWD=mypwd

Then, run the azkaban-installer.jar with the ‘the upgrade-schema’ command:

Code Block
java.exe -jar "azkaban-installer.jar" upgrade-schema --database=AZ_REPO --host=PG_HOST --port=5432 --schema=white --username-var=AZ_DB_USER --password-var=AZ_DB_PWD

...

To start using this scheduler in RED first, the Scheduler Configuration needs to be added to RED via the Scheduler tab.

...

Enter your URLs for the Azkaban Web Server you setup earlier into both the Dashboard and Rest API URL fields.
Provide the credentials for Dashboard User into the Dashboard section
Provide the credentials for Publish User in the Rest API section
Add your default scheduler tags which you setup on your Executor servers (these are case sensitive so ensure you have the same case as in your response file)
Set the default scripts as shown in the example below

...

Image Added

Example response files:

Note

title	Note

...

You must install the Azkaban Web Server first

...

before installing any Executor Servers.

...

The Web Server and Executor Servers can be installed on Windows or Linux, or some combination of both

...

. It is important to ensure that connectivity between your Web and Executor Servers is open on the ports you assign to them.

Code Block

language	xml
title	Web Server Response File
collapse	true

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<installServerResponse>
    <serverType>WEB_SERVER</serverType>
    <serverName>WEB_SERVER</serverName>
    <destination>
        <destinationDirectory>\home\AzkabanSchedulerUser\WhereScape\</destinationDirectory>
    </destination>
    <metadataDatabase>
        <databaseName>schedmeta</databaseName>
        <hostName>pgserver</hostName>
        <password>schedpassword</password>
        <port>5432</port>
        <schema>white</schema>
        <userName>scheduser</userName>
    </metadataDatabase>
    <redMetadataDatabase>
        <databaseName>redmeta</databaseName>
        <hostName>pgserver</hostName>
        <password>redpassword</password>
        <port>5432</port>
        <schema>red</schema>
        <userName>reduser</userName>
    </redMetadataDatabase>
    <serverConfig>
        <executorTags></executorTags>
        <port>8001</port>
        <timeZone>Pacific/Auckland</timeZone>
        <webServerExternalHostName>AzkabanServer</webServerExternalHostName>
        <webServerExternalPort>8001</webServerExternalPort>
        <adminUserName></adminUserName>
        <adminPassword></adminPassword>
        <dashboardUserName>wsdashboarduser</dashboardUserName>
        <dashboardPassword>wsdashboardpass</dashboardPassword>
        <publishUserName>wsapiuser</publishUserName>
        <publishPassword>wsapipass</publishPassword>
    </serverConfig>
    <emailConfig>
        <host>emailhost</host>
        <port>587</port>
        <user>emailuser</user>
        <password>emailpassword</password>
        <sender>senderemail</sender>
        <tls>true</tls>
        <useAuth>true</useAuth>
    </emailConfig>
    <redBindir>\home\AzkabanSchedulerUser\WhereScape\</redBindir>
	<azkabanPasswordEncryption>NONE</azkabanPasswordEncryption>
	<azkabanWalletCmd></azkabanWalletCmd>
	<redProfilePassword>redprofilepass</redProfilePassword>
	<wherescapeJobWorkdir>/tmp</wherescapeJobWorkdir>
</installServerResponse>

...

Code Block

language	xml
title	Executor Server Response File
collapse	true

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<installServerResponse>
    <serverType>EXEC_SERVER</serverType>
    <serverName>EXEC_SERVER</serverName>
    <destination>
        <destinationDirectory>\home\AzkabanSchedulerUser\WhereScape\</destinationDirectory>
    </destination>
    <metadataDatabase>
        <databaseName>schedmeta</databaseName>
        <hostName>pgserver</hostName>
        <password>schedpassword</password>
        <port>5432</port>
        <schema>white</schema>
        <userName>scheduser</userName>
    </metadataDatabase>
    <redMetadataDatabase>
        <databaseName>redmeta</databaseName>
        <hostName>pgserver</hostName>
        <password>redpassword</password>
        <port>5432</port>
        <schema>red</schema>
        <userName>reduser</userName>
    </redMetadataDatabase>
    <serverConfig>
        <executorTags>
			<tag>linux</tag>
		</executorTags>
        <port>8002</port>
        <timeZone>Pacific/Auckland</timeZone>
        <webServerExternalHostName>AzkabanServer</webServerExternalHostName>
        <webServerExternalPort>8001</webServerExternalPort>
        <adminUserName></adminUserName>
        <adminPassword></adminPassword>
        <dashboardUserName></dashboardUserName>
        <dashboardPassword></dashboardPassword>
        <publishUserName></publishUserName>
        <publishPassword></publishPassword>
    </serverConfig>
    <emailConfig>
        <host>emailhost</host>
        <port>587</port>
        <user>emailuser</user>
        <password>emailpassword</password>
        <sender>senderemail</sender>
        <tls>true</tls>
        <useAuth>true</useAuth>
    </emailConfig>
    <redBindir>\home\AzkabanSchedulerUser\WhereScape\</redBindir>
 	<azkabanPasswordEncryption>NONE</azkabanPasswordEncryption>
	<azkabanWalletCmd></azkabanWalletCmd>
	<redProfilePassword>redprofilepass</redProfilePassword>
	<wherescapeJobWorkdir>/tmp</wherescapeJobWorkdir> 
</installServerResponse>

RED has implemented two password values for the azkabanPasswordEncryption property for Linux:

UseWALLET value for azkaban.walletCmd which is used as a system command to retrieve the password from the wallet. For more information on Wallet configuration using a standard Linux password manager, visit https://www.passwordstore.org/
Or, use NONE

For more information about wherescapeJobWorkdir property, refer to RED Scheduler Configuration.

`Example Install Command:`

Typical installation command assuming the azkaban-installer.jar has been copied to the home directory for the running user on Linux:

...

`Starting and Stopping the Azkaban Servervs`

On Linux, the Azkaban Servers are not started automatically after installation to run the startup script.