When the user runs the WhereScape CICD Enablement Pack, this will start the process of connecting to GIT and building the WhereScape 3D and RED repositories.

This should be done at the following times: 

1. When connecting to a new GIT repository. 
2. When connecting to an existing GIT repository for the first time (on a Developer machine). 
3. When refreshing a Personal Access Token to an existing GIT repository. 
4. When rebuilding a corrupted Developer machine's GIT repository. 

When the process is launched, it will prompt the user for the following information:

• Select the repository types: Select the product(s) you want to utilize from the check buttons, choosing RED, 3D or both. 
• Environment Type: Developer Machine or Deployment Machine to be selected from the dropdown menu.
  • Developer Machine – Used for development activities
  • Deployment Machine - Used for CICD Workflows 
• Environment Tag: One letter can be entered here to provide a description for what the environment is used for, For example ‘d’ for development, or ‘t’ for testing. This will be included in the Repository name. 
• On Pull Action: Drop Schema or Drop Database to be selected from the dropdown menu. One of the following will hapepen when the user initiates a pull:
  • Drop Schema – Removes all schemas on target PostgreSQL Repository
  • Removes entire target PostgreSQL Repository

  On all pull requests the default structure and metadata will be rebuild using the WhereScape CLI.

• GIT URL: Enter the GIT URL for this repository e.g. github.com. 
• GIT Organization: Enter the organization name used for the GIT Repository.
• GIT User Email: Enter the User's email for the GIT Repository.
• GIT Repository: Enter he GIT Repository name (remote) to connect to. 
• Username: Enter the user's Username to be used for this GIT Repository.
• Personal Access Token: Enter the user's Personal Access Token for this GIT Repository. 
• Local GIT Repository: The location of the GIT repository created locally on the machine/virtual machine will be displayed (read-only). 
• Build Options: Select ‘Connect to Existing CICD GIT Repository’ if there is existing metadata in the GIT Repository. Selecting ‘Initial Build’ will deploy the initial CICD Enablement Pack metadata and configure WhereScape with CICD functionality. For both options, Metadata will be deployed to the WhereScape PostgreSQL repositories. Users may want to use an initial Build for the following scenarios:
  • First-Time GIT Integration 
    • Setting up a new WhereScape repository with GIT integration 
    • Configuring initial repository settings and workflows 
    • Establishing protected branches and default configurations 
  • Repository Import 
    • Importing an existing WhereScape repository backup 
  • Repository Recovery 
    • Reinitializing a corrupted repository 
    • Restoring from known good configuration 
    • Re-establishing GIT integration after issues 

  • Upgrading your Enablement Pack

    • When you update your CICD Enablement Pack, running an initial build will ensure all necessary enablement pack scripts are up to date 

• RED Repository Server and Port: Add the RED Repository PostgreSQL Server hostname and port. 
• RED Repository Database Name: The repository database name will be calculated based on the information provided – this is read-only. This can have a max character length of 63 characters. If you would like to change the way this is configured, please consult section 'Changing the Repository Database Name'. 
• RED Repository Username and Password: Add the RED Repository PostgreSQL Username and Password with permissions to the specified database
• RED Generate Files: Use the check buttons to select whether you would like to generate DDL, DML and/or JSON files when committing to GIT. These options are only available when RED is selected and Build Option is set to ‘Connect to existing CICD GIT Repository’. 
• RED Repository Backup: If Build Options is set to ‘Initial Build’ the user will be required to select a backup file to be migrated to their GIT Repository. This is mandatory for any Initial Build with RED enabled.
  

  The backup files will need to be in SQL, or BAK file format.  The PostgreSQL version used for the backup must match the version of the target repository.  For full information on creating and using a backup, please consult section ‘WhereScape 3D and RED Backup Files’.

• 3D Repository Server: Add the 3D Repository PostgreSQL Server and port.

• 3D Repository Database Name: The repository database name will be calculated based on the information provided – this is read-only. This can have a max character length of 63 characters. If you would like to change the way this is configured, please consult section 'Changing the Repository Database Name'.

• 3D Repository Username and Password: Add the 3D Repository PostreSQL Username and Password with permissions to specified database.

• 3D Repository Backup: If ‘Build Options’ is set to ‘Initial Build’, the user will have the opportunity to select a backup file to be migrated to their GIT Repository. This is optional. If a user does not select a backup file, a new blank repository will be created.
  

  The backup files will need to be in SQL, or BAK file format.  The PostgreSQL version used for the backup must match the version of the target repository.  For full information on creating and using a backup, please consult section ‘WhereScape 3D and RED Backup Files’.

   

• Once the form is created click on 'OK'. If there are any issues with connecting to the GIT repository, the user will receive an error message.

• The local GIT directory will now be created.

• The user credentials for WhereScape RED and 3D repositories are created:
  • These are stored locally at this location: `%USERPROFILE%\WhereScape\GIT`. 
  • This file stores the user's credentials and configuration details which are required for the workflows to run correctly. 
    

    Each GIT Repository will have its own profile file. If it is deleted or modified incorrectly, the workflows will stop working.

• The user will be prompted to select a branch from the remote GIT repository.
  
   

• This action refreshes the local GIT repository with updates from the specified remote branch.

• WhereScape 3D and/or RED is rebuilt using the files within the local GIT repository. 

If the user requires to connect to a new or different GIT repository, these steps should be repeated using the new settings.

Changing the Repository Database Name

When completing the WhereScape CICD Enablement Pack form, the RED and 3D Repository Database names are read-only and are created based on the user information provided elsewhere on the form. However, this can be changed via the config file if required. 

1. The config is held in a JSON file named 'config.json' at the following location: 
    %USERPROFILE%\WhereScape\GIT 
2. The repository database names are calculated within their resepective RED or 3D sections under the 'Repository Database name prefix and suffix' keys. These values can be amended to change the way that the repository database names are created when completing the WhereScape CICD Enablement Pack from. 
   
   

WhereScape 3D

By default, WhereScape 3D and RED store their metadata in separate PostgreSQL databases. To make sure both applications stay in sync, GIT is used as the master source for all metadata. 

When developers pull the latest metadata files from the remote GIT repository, their local WhereScape repositories are rebuilt using those files. This keeps everything aligned to the same version (called a commit ID) and ensures consistency across both tools. 

This process has been designed to ensure that both WhereScape repositories are always aligned. 

Using WhereScape 3D Workflows

To use WhereScape GIT Enablement Pack workflow scripts in WhereScape 3D, users will need to access their workflows via the Repository, Model or Model Version on the Repository Browser panel. When clicking into a Repository, Category, Model or model version, users can view the workflow tiles within the workflow manager. 

Different workflow scripts are accessible from various items within the Repository panel. For detailed information on each script, including its location and usage, please refer to the section titled ‘Using WhereScape GIT Enablement Pack Workflows’. 

To configure your workflows, please consult section ‘Configuring Workflows in WhereScape 3D’. 

All existing WhereScape 3D workflow processes remain available.

Configuring Workflows in WhereScape 3D

Users will find the GIT integration tools within the Workflow pane. Depending on the type of model you are currently viewing, there may be different buttons available by default. 

All existing WhereScape 3D workflow functions remain available. However, a maximum of 12 functions can be visibile at a time, so users with pre-existing Workflow set-ups may need to manually reconfigure the Workflow to pick and choose which functionality they want to keep or replace. 

Users can update their Workflow sets by doing the following: 

1. Click on Tools > Manage Workflows.
    
2.  The user will be presented with a list of customer model versions. Click on a version to view the Workflow configuration in the panel to the right. 
   
3. The user can either create a new Workflow set from scratch, or edit an existing Workflow set. 
   1. To create a new Workflow set the user can click on the 'Add Workflow Set' icon. Give the workflow set a name, select the Type from the dropdown and select 'OK'. 
       
   2.  To edit an existing Workflow set, the user can highlight the set they would like to edit by clicking on it, to bring up the Workflow configuration panel. 
4. Users can add new buttons by clicking on the 'Add Button' icon positioned above the Workflow configuration panel. 
   
5.  The user can then provide the details of their Workflow: 
   1. Provide the name for the workflow button you are adding. 
   2. Provide a description for the workflow button (optional) 
   3. Select an Icon to represent your workflow button. 
   4. Select the Type of workflow item you want to add. All GIT integration Workflows are kept as type 'Script'. 
   5. Select the script from the dropdown which represents your required Workflow. 
   6. Once you are happy with your Workflow button, click 'Apply'. 
      
6.  The user will see their new Workflow button in the Workflow configuration panel.
    
7.  Users can reposition or delete any workflow buttons by right-clicking on a button and choosing the required action. 
   
8.  Once the user is happy with their configuration, they can click 'OK'. 

3D FAQs 

Conflicts when merging branches 

The Development Lead creates two feature branches, one for Developer A and one for Developer B. 

Monday: Developer A completes their development within their feature branches, then merges the feature branch into Development. 

Tuesday: Developer B completes development within their feature branches and then merges the feature branch into Development. This merge fails because both developers have made changes to the same model, resulting in a complex change that cannot be automatically merged. In other words, there is a lack of understanding about which changes should be kept, and which should be overwritten. 

Therefore, during the merge and close function, if any conflicts are found, the user will receive a prompt to either discontinue progress on the script or view the conflicts. 

If the users choose to review the conflicts they will be prompted with a list of the conflicted files and the conflicts highlighted within them.

It will then be up to the user to resolve the conflicts outside of the enablement pack before trying to the Merge & Close again. 

WhereScape RED

By default, WhereScape RED’s metadata is held in a separate PostgreSQL database from WhereScape 3D. 

GIT holds the master version of the WhereScape metadata. Therefore, to keep the Developer machine WhereScape repositories aligned, they are rebuilt from GIT after the metadata files are pulled directly from the remote GIT repository. 

This ensures that both WhereScape applications are built to the same commit ID, ensuring the integrity of the data models & WhereScape application configuration across both applications. 

Using WhereScape RED Workflows

In WhereScape RED, all workflows are accessible from the Development panel by right clicking on ‘All Objects’ > ‘Workflows’. 

Users will then see a list of all available workflow scripts to choose from. 

For detailed information on each script, please refer to the section titled ‘Using WhereScape GIT Enablement Pack Workflows’. 

Using WhereScape GIT Enablement Pack Workflows

Committing to GIT

WhereScape 3D Only- When using WhereScape RED, users are currently only able to ‘Commit ALL Objects to GIT’. 

To commit a new model or changes to an existing model to GIT, users click the workflow button, which executes the ed3d_commit_to_git template.

By default, this script can be activated by the Repository, Source Category and Model Version level of the Repositories panel. 

This workflow is used for the following reasons:

1. After making a change to a new data model.
2. After making a change to an existing data model.

This workflow will perform the following tasks:

1. Verifies connection to GIT Repository.
2. Brings in the latest changes from the remote repository so your local setup is up to date.
3. Saves the current version of your local repository.
4. The user will be prompted to add their own commit message:
    
   • If the user would like to link their commit to a backlog item or bug in GIT, they can use development tags in their commit message. Simply add a '#' followed by the backlog item ID. For example: '#123', will automatically associate the commit with the corresponding work item in your backlog.
5. Commits your changes to the remote GIT repository with your message.

Users can create/change the workflow configuration for any category, model, or model version; however, this function requires the user to select the model version to operate correctly.

Committing ALL Objects to GIT

Available in WhereScape 3D and RED

To commit all changes made to a WhereScape GIT Repository, users click the 'Commit All Objects to GIT' workflow button in WhereScape 3D or RED, which executes the ed3d_commit_all_objects_to_git template.

In WhereScape 3D by default, this script can be activated from the Repository or Category level of the Repositories panel.  

This workflow is used for the following reasons:

1. After making changes to multiple new and/or existing data models.
2. After making changes to data models and configuration.

This workflow will perform the following tasks:

1. Verifies connection to GIT Repository.
2. Brings in the latest changes from the remote repository so your local setup is up to date.
3. Saves the current version of your local repository.
4. The user will be prompted to add their own commit message:
   
   • If the user would like to link their commit to a backlog item or bug in GIT, they can use development tags in their commit message. Simply add a '#' followed by the backlog item ID. For example: '#123', will automatically associate the commit with the corresponding work item in your backlog.
     
     
5. Commits all your changes to the remote GIT repository with your message.

Committing configuration changes to GIT

WhereScape 3D Only - When using WhereScape RED, users are currently only able to ‘Commit ALL Objects to GIT’. 

To commit any configuration changes to GIT, users will click the workflow button, which executes the ed3d_commit_config_to_git template.

By default, this script can be activated by the Source Category level of the Repositories panel. 

This workflow is used for the following reasons:

1. After creating a new configuration setting.
2. After deleting an existing configuration setting.
3. After changing an existing configuration setting.

This workflow will perform the following tasks:

1. Verifies connection to GIT Repository.
2. Brings in the latest config changes from the remote repository so your local setup is up to date.
3. Refers to the GIT Audit Change log within the WhereScape 3D repository.
4. Extracts the configuration files & stores them within the local GIT repository folder.
5. The user will be prompted to add their own commit message:
   
   • If the user would like to link their commit to a backlog item or bug in GIT, they can use development tags in their commit message. Simply add a '#' followed by the backlog item ID. For example: '#123', will automatically associate the commit with the corresponding work item in your backlog.
6. Commits the config change to the remote GIT repository with your message.

Delete Model and Commit Changes

WhereScape 3D Only

To delete a model from your repository and commit the changes, users will click the workflow button 'Delete Model and Commit', which executes the ed3d_delete_model_commit_to_git template.

By default, this script can be activated by the Model Version level of the Repositories panel. 

This workflow is used for the following reasons:

1. When a model is no longer required and needs to be removed.

This workflow will perform the following tasks:

1. Deletes the currently selected model.
2. Verifies connection to GIT Repository.
3. Brings in the latest changes from the remote repository so your local setup is up to date.
4. Saves the current version of your local repository.
5. Commits the change to the remote GIT repository.

Create Branch

Available in WhereScape 3D and RED

To create a new branch for your WhereScape GIT Repository, users click the Create Branch workflow button in WhereScape 3D or RED, which executes the ed3d_create_feature_branch template. 

In WhereScape 3D by default, this script can be activated from the Repository or Source Model level of the Repositories panel.  

This workflow is used for the following reasons:

1. Preparing a copy of an existing branch for a new purpose/development task/developer.

This workflow will perform the following tasks:

1. Verifies connection to GIT Repository.
2. Brings in the latest changes from the remote repository so your local setup is up to date.
3. The user will be prompted with a window and asked to provide the following information:
   1. The branch they wish to clone.
      1. As of WhereScape CICD Enablement Pack version 1.4.0.0, the user can select ‘Local Repository’ which will take any changes made since the last commit and clone them into a new branch. The initial branch will revert back to its last committed version.  
   2. The name the new branch.
   3. Click 'Create'
      
4. WhereScape 3D is closed.
5. The selected branch will be cloned on the remote GIT repository.
6. The new branch will be pulled onto the local\virtual machine.
7. WhereScape 3D is rebuilt using the files within the local GIT repository.
8. WhereScape RED (if using) is rebuilt using the files within the local GIT repository.
9. WhereScape 3D is reopened.

Close Branch 

Available in WhereScape 3D and RED

To close an existing branch for your WhereScape GIT Repository, users will click the 'Close Branch' workflow button, which executes the ed3d_close_branch template. 

This workflow is used for the following reasons: 

1. Close and discard the branch without merging changes into another branch. 
2. Merge the current branch into another branch (e.g. feature branch to be merged into development). 

This workflow will perform the following tasks:

1. Verifies connection to GIT Repository. 
2. The user is prompted to select the branch to close. The user can close a branch and discard all changes or close and merge the branch into another branch. The user will choose a branch to switch or merge to. 
   
3. If the user selects to close and merge, then a GIT merge action will take place between the selected branches. If there are any conflicts found during the merge, the user will receive a prompt to either discontinue progress on the script, or to view the conflicts: 
   
   1. If the user cancels, the script will be exited.  
   2. If the users chooses to review the conflicts they will be prompted with a list of the conflicted files and the conflicts highlighted within them.
      
      
   3. It will then be up to the user to resolve the conflicts before trying to the Merge & Close again. 
4. The WhereScape app is closed.
5. The local GIT repository is refreshed from the selected branch on the remote GIT repository. 
6. WhereScape 3D is rebuilt using the files within the local GIT repository. 
7. WhereScape RED (if using) is rebuilt using the files within the local GIT repository. 
8. WhereScape 3D is reopened. 

Switch Branch 

Available in WhereScape 3D and RED

To switch to a different branch within your WhereScape GIT Repository, users will click the ‘Switch Branch’ workflow button, which executes the ed3d_switch_branch template. 

In WhereScape 3D by default, this script can be activated from the Repository or Source Model level of the Repositories panel.  

This workflow is used for the following reasons:

1. To review the code/data model of the alternative branch.
2. Start development on a different branch.

This workflow will perform the following tasks:

1. Verifies connection to GIT Repository.
2. The user is prompted to select the branch to switch to.
   
3. Refreshes the local GIT repository from the remote GIT repository of the selected branch.
4. WhereScape 3D is rebuilt using the files within the local GIT repository.
5. WhereScape RED is rebuilt using the files within the local GIT repository.
6. WhereScape 3D is reopened.

Pull from GIT

Available in WhereScape 3D and RED

To fetch the latest changes for the remote GIT Repository, users will click the ‘Pull’ workflow button, which executes the ed3d_pull template. 

In WhereScape 3D by default, this script can be activated from the Repository or Source Model level of the Repositories panel. 

This workflow is used for the following reasons:

1. Refresh the Developer machines WhereScape repositories with the current (commit) metadata on the remote GIT repository.
2. Refresh and rebase the Developer machines WhereScape repositories, discarding any uncommitted changes.

This workflow will perform the following tasks:

1. Verifies connection to GIT Repository.
2. The “Branch” stored within the WhereScape Repository is the selected branch from which to pull.
3. Refresh the local GIT repository with updates from the corresponding remote GIT repository.
4. WhereScape 3D is rebuilt using the files within the local GIT repository.
5. WhereScape RED is rebuilt using the files within the local GIT repository.
6. WhereScape 3D is reopened.

Export to RED

WhereScape 3D Only

To export a RED export model WhereScape RED, users will click the 'Export to RED' workflow button, which executes the ed3d_git_to_red templates.

By default, this script can be only be activated on a ‘RED export model version’ in the Repositories panel. 

This workflow is used for the following reasons:

1. Deploy a new Data Model into the WhereScape RED repository.
2. Deploy changes to an existing Data Model into the WhereScape RED repository.

This workflow will perform the following tasks:

1. Verifies connection to GIT Repository.
2. Brings in the latest changes from the remote repository so your local setup is up to date.
3. Saves the current version of your local repository.
4. Commits the change to the remote GIT repository.
5. You will be prompted to add a commit message:
   
6. Deployment of the model to RED completes.

Import Model

WhereScape 3D Only

To import a Data Model from a selected commit ID from a branch, users will click the 'Import from GIT' workflow button. This workflow button executes the ed3d_import_model_from_git template.

This workflow is used for the following reasons:

1. Import merge conflicts from a different branch.
2. Import a historical version of a data model into the existing branch.

This workflow will perform the following tasks:

1. Verifies connection to GIT Repository.
2. A window opens prompting the user for the following information:
   1. Select a branch
   2. Select a commit reference you wish to import from.
      Please note: These commits are filtered only to show commits that include the selected data model version.
   3. The user can review the commit changes they have selected in the text box.
   4. The user can overwrite the selected model version OR create a new version, using the commit ID as a prefix to the current model version name.
   5. Select 'OK'
      

3. The local GIT repository is synchronized to the selected commit.

4. A new model version is created, and the model is imported to your local repository.

5. The user must refresh the Repository Browser to see the imported data model.

Update Metadata 

Available in WhereScape 3D and RED

To update and commit the metadata in your repository, users will click the workflow button ‘Update Metadata’, which executes the edws_update_metadata template. 

In WhereScape 3D by default, this script can be activated from the Repository level of the Repositories panel.  

This workflow is used for the following reasons: 

1. When a user has updgraded to the latest version of WhereScape GIT Enablement Pack and needs to update the template scripts. 

This workflow will perform the following tasks: 

1. Verifies connection to the GIT Repository. 
2. Checks for uncommitted changes and displays the following warning: “Uncommitted changes in your local repository will be lost when updating metadata”. 
3. Closes WhereScape application 
4. Copy the metadata files from the enablement pack to the local GIT Repository on the user’s branch. 
5. Commits the updated metadata files to GIT. 
6. Resets the WhereScape Repository. 
7. Reopens WhereScape application  

Upgrades

When the MSI is required to be updated, two options can be considered:

Option 1

Use this for partially complete development activities or code not ready to merge to the parent branch.

1. Developer A is connected to a feature branch. Developer B is connected to the development branch.
2. Both developers should commit our changes back to the branch they are connected to, ensuring their changes are within the GIT repository.
3. The MSI is updated on all developer machines.
4. Each developer would either:
   1. Use the desktop shortcut to reconnect to the GIT repository and their branch, or
   2. Open WhereScape 3D and click the "Pull" workflow button.

Option 2

This option can be used if all development activities are complete. It is not recommended for partially completed development activities.

1. Developer A creates a feature branch for Developer B.
2. Developer B is connected to a feature branch.
3. Developer B completes development activities.
4. Developer A merges the feature branch into the development branch.
5. The MSI is updated on all developer machines.
6. Developer A would either:
   1. Use the desktop shortcut to reconnect to the GIT repository and development branch, or
   2. Open up WhereScape 3D and click on the "Pull" workflow button.
7. Developer A creates a feature branch for Developer B.
8. Development activities continue.

In either option, the files from the remote GIT repository are pulled to the local machine, which is then used to reconfigure the local environment and the PostgreSQL repository used by WhereScape RED and 3D.

Other Considerations

The following list are known issues:

WhereScape 3D

• Importing XML Model Version files that contain custom UI source connections. This results in Command line performance issues – memory leakage when running parallel tasks.
• RED Discovery & profiling connection is not compatible with the CICD/GIT functions at present. Users can create this separately, but it must be manually refreshed.
• Target Storage Locations are added to WhereScape RED via the Python Scripts, not the WhereScape command line.
• It is possible to create duplicate Data Type names. However, this will produce an error when the command line interacts with the Data Types (duplicated name).
• Templates: Templates with XML contents can result in an error when imported/exported using the command line.

WhereScape RED

• It is not possible to change the Script Launcher options and commit them to GIT. This is only possible as a change request as it is possible within the Python Scripts.
• Passwords can be stored within WST. It is recommended that any new repositories are set NOT to store passwords within WST. Within WhereScape RED, go to the Options to make these changes.
• WST File – If a table or view object contains a duplicate column name, it is possible to commit these changes to GIT; however, it will produce an error when deploying the WST file.
• If the initial build fails and the log file returns an error relating to the License Agreement, open WhereScape RED and agree to the license agreement when prompted.
• PostgreSQL command line
• Ensure that the command line applications, found in `C:\Program Files\WhereScape\RED\Tools` are kept in line with the version of the PostgreSQL database being used to host the WhereScape RED & 3D applications.
• It is not recommended to mix versions of PostgreSQL across WhereScape applications.

GIT

• File size is limited to 50 MB. AS WST files can exceed this limit, the Python script compresses the WST files into a ZIP file (binary) before committing to GIT. Other files may exceed this limit in the future.
• Ensure that the GIT runner folder allows the account executiong the Python scripts with Full Control permissions. Failure to do this, may result in the WST deployment script from working correctly.
• Ensure that the default branch within the global gitconfig file (`C:\Program Files\Git\etc\gitconfig`) is set to the default branch within the Python config settings. This will avoid issues when cloning the remote GIT repository to the Developer machine. This must be completed with administration privileges.
  • For example:
    [init]
defaultBranch = main[safe]
directory = *