Once an EDW 3NF object has been defined in the metadata and created in the database, an update procedure can be generated to handle the joining of any tables and the update of the EDW 3NF object.

Generating a Procedure

To generate a procedure:

  1. Right-click the EDW 3NF object in the left pane and select Properties.
  2. Click the Rebuild button to start the process of generating the new procedure.
  3. A series of options are available.

Processing Tab

OptionsDescription
TemplateEnables you to generate update procedures via a template.
Business Key Columns
Columns that define the business key for update processing. Required to include Update options. Clicking the ellipsis button brings up the Business Key selection screen.

A business key can be made up of multiple columns, but it must provide a unique identifier. Where multiple columns separately uniquely identify rows in the EDW 3NF Object, choose one to act as the primary business key. For example, a source table may have a unique constraint on both a product code and a product description. Therefore the description as well as the code must be unique. It is possible to combine the two columns, but the normal practice would be to choose the code as the business key.
  • Use the column name ascending/descending buttons to sort column names. To revert to the meta column order, click on the meta column order button.
  • NULL Values - none of the columns chosen as the business key should ever contain a NULL value.


Parameters
Any parameters selected are included in the generated update procedure as variables. The procedure includes code to retrieve the value of the parameter at run time and store it in the declared variable.
Clicking the ellipsis button brings up the Parameters selection screen.

The variables can also be used in column transformations and in the from/where clause for the update procedure. Some databases have a 30 character limit for variable names. WhereScape RED ensures the variables added for any parameters are less than 30 characters long by creating variable names in the form v_ followed by the first 28 characters of the parameter name.
For example, a parameter called MINIMUM_ORDER_NUMBER_SINCE_LAST_SOURCE_LOAD will be available as the variable v_MINIMUM_ORDER_NUMBER_SINCE_L.
  • WhereScape RED parameters should be unique within the first 28 characters to avoid conflicting variables names.
  • If the desired parameter doesn't exist in the metadata yet, a new parameter can be added by clicking on the Add New button on the bottom leftmost corner of the Select Parameters dialog.
  • Refer to Parameters for details.


Include Initial Load InsertAdds insert statement to the update procedure that runs if the target EDW 3NF object is empty. The benefit of this is improved performance inserting into an empty table without performing any checks to see if rows already exist. The default for this field is off (i.e. an initial insert statement is not added to the procedure).
Process by BatchEnables you to select a column to drive data processing in a loop based on the distinct ordered values of the selected columns. The update procedure loops on this column and performs the delete, update and/or insert for each value. If the column chosen is a date data type (date, datetime or timestamp), then the user can specify yearly, monthly, daily or column level looping. The default for this field is off (do not do batch processing).
Delete Before InsertEnables you to select how to process deletes. It enables a delete statement to be added to the update procedure before any update or insert statement. This is a particularly useful option for purging old data and for updates based on a source system batch number. If this option is selected, the following options are also available:

  • Issue Warning if a Delete occurs: this option sets the procedure to a warning state if any deletes occur.

  • Delete Where Clause: the delete where clause is appended to the generated delete statement to constrain the rows deleted.

Process MethodSelect between Insert/Update and Merge which allows you to use the Merge statement instead of two separate Insert and update statements.
Include Insert Statement

Includes the insert statement in the procedure to insert new rows in the EDW 3NF Object.

Insert

Enter a database compliant hint to be used in the INSERT statement. This is a SQL Server only option. Defaults can be configured in Tools > Options > Default Update Procedure Options.

Insert New Rows OnlyUses change detection to work out what rows require inserting.
New Row Identification MethodMethod used to identify that records in the source are not currently recorded in the target table.
Existing Data Selection HintDatabase-compliant hint to be used for the existing data select statement.
Include Update Statement

Includes an update statement in the procedure to update changing rows in the EDW 3NF Object. If this option is chosen, then the Update Changed rows only option is available.

Update

Enter a database compliant hint to be used in the UPDATE statement. This is a SQL Server only option. Defaults can be configured in Tools > Options > Default Update Procedure Options.


Update Changed Rows OnlyUses change detection to work out what rows require updating. Choosing this option, enables the Change Row identification Method.
Change Row Identification MethodMethod used to identify that records in source have changed from what is currently recorded in the target table.
Existing Data Selection Hint

Database-compliant hint to be used for the existing data select statement.

Merge

Enter a database hint to be used in the MERGE statement. This is a SQL Server only option. Defaults can be configured in Tools > Options > Default Update Procedure Options.


Source Tab

OptionsDescription
Distinct Data SelectEnsures duplicate rows are not added to the EDW 3NF Object. This is achieved by the word DISTINCT being added to the source select in the update procedure. This is not selected by default. Select Hint: A database-compliant hint to be used in the source Select statement.
Source JoinThe From clause, including Source Join information.
Where ClauseThe Where clause.
Group ByThe Group By clause.

Simple Join

A simple join only returns rows where data is matched in both tables. So for example, if table A has 100 rows and table B has a subset of 24 rows. If all the rows in table B can be joined to table A then 24 rows will be returned. The other 76 rows from table A will not be returned.

Outer Join

The outer join returns all rows in the master table regardless of whether or not they are found in the second table. So if the example above was executed with table A as the master table then 100 rows would be returned. 76 of those rows would have null values for the table B columns.

  • When WhereScape RED builds up an outer join, it needs to know which table is the master table and which is subordinate. Select the join column from the master table first. In the example screen above the table 'load_order_header' has had its column chosen and the column for the table 'load_order_line' is currently being chosen. This will result in the 'load_order_header' table being defined as the master, as per the example statement above. The results of this example select are that a row is added containing order information, regardless of whether or not a corresponding load_order_line entry exists.
  • When upgrading from a RED version previous to 6.8.2.0 and moving existing objects to a target location, all procedures that reference those objects will need to be rebuilt.
    Any FROM clauses will also need to be manually regenerated for the table references to be updated to the new [TABLEOWNER] form.

Building and Compiling the Procedure

  • Once the relevant options are completed, click OK. The procedure is built and compiled.
  • If the compile fails an error is displayed along with the first few lines of error messages. Compile fails typically occur when the physical creation of the table was not done.
  • If the compile fails for some other reason the best approach is to use the procedure editor to edit and compile the procedure. The procedure editor highlights all the errors within the context of the procedure.
  • Once the procedure has been successfully compiled, it can either be executed interactively or passed to the scheduler.

Indexes

By default, a number of indexes will be created to support each EDW 3NF table. These indexes are added once the procedure has been built.
An example of the type of indexes created is as follows:

Additional indexes can be added, or these indexes changed. Refer to Indexes

for more details.

Converting an existing EDW 3NF Table to an EDW 3NF History Table

To convert an EDW 3NF table to an EDW 3NF history table:

  • Change the table type to History in the Properties screen.
  • Select (Build Procedure...) from the Update Procedure drop-down list and click OK.

If the existing EDW 3NF table is NOT to be dropped and recreated, then the following process must be followed:

  1. Click OK in the Procedure Type window.
  2. Click OK in the Business Key window.
  3. Click Alter in the Adding Additional Columns window.

  4. Click Alter Table on the Alter Table Commands window:

    The SQL statement in this window can be edited if required.

  5. Click OK on all remaining windows.


  • No labels