9.218 TRANSFORM_FILE

Transforms the current contents of a disk file into one or more working lists.

It is designed to facilitate the transfer of information between Visual LANSA applications and other products such as spreadsheets.

Arguments

Return Values

Technical Notes - TRANSFORM_FILE

File Name

Refer to TRANSFORM_LIST's 9.219.1 Output File Formats for detailed information.

Input File Format

Refer to TRANSFORM_LIST's 9.219.1 Output File Formats for detailed information.

The special options CL and DL are available for testing and debugging purposes only. Do not use these options in production software. When used, these options do not attempt to read the nominated lists. Instead, the layout details are prepared, and the expected layout of the input file is actually printed back into the file specified in the file parameter. If the nominated file does not exist, it is created. If it does exist, then all existing data in the file is erased before the layout details are printed into it. Once the layout file has been produced it can be reviewed by any standard text editor. This information allows you to compare the actual input data file record layout with what this Built-In Function expects the layout to be. Such a comparison can be used to locate formatting problems.

Invalid Character handling in Alpha, Char , String, BLOB, CLOB Fields

Refer to Invalid Character handling in Alpha, Char , String, BLOB, CLOB Fields in TRANSFORM_LIST's 9.219.1 Output File Formats . The same rules apply, but to fields in the working lists rather than to those in the output files.

There are some special characters which cannot be processed properly by this Built-In Function. These are: Carriage Return (binary value in Windows: 0x0D), Line Feed (binary value in Windows: 0x0A) and CTRL+Z (binary value in Windows: 0x1A). For example, TRANSFORM_FILE will stop reading the file when it encounters '0x1A' character in Windows.

The best way to output binary data to a file by TRANSFORM_LIST is to convert it into alphanumeric strings by using the Built-In Function 9.9 BINTOHEX, then output the result by TRANSFORM_LIST.

To retrieve the data back, use TRANSFORM_FILE to read the data then use 9.115 HEXTOBIN to convert the data back to the original format.

Decimal Point Character

Normally, this option is only required by customers, in some European countries, who have configured their systems to use a comma ( , ) as the decimal point delimiter. By default the Built-In Function will expect a comma. This may not be appropriate when using data output by other products that do not produce the comma. This option may be used to force this BIF to expect the full stop/period character ( . ).

Its other use involves the special R (remove) option that may be used when reading files created by applications that use fixed record formats and "implied (by position) " decimal points. In this case the number of decimal positions is implied by the definition of the receiving field in the working list. Refer to the TRANSFORM_LIST command for more details.

The R option can only be used with file formats C and D.

Close Input File Option

This option prevents this Built-In Function from closing the input file when it has completed execution, unless the end of the file is encountered.

In normal use, a call is made to this Built-In Function, it clears the lists, loads as much data as will fit into the lists, closes the input file and returns control to the invoking function. The return code will be set to "EF" or "OV" indicating whether the entire input file would fit into the list.

However, by using the "do not close" option it is possible to perform more complex processing such as:
Reading input files that have more than 9999 records and/or avoid making huge working lists, which require a large amount of allocated memory.

     def_list #list fields(....) listcount(#count)
              type(*working) entrys (100)
     dowhile (#retcode *ne EF)
          use TRANSFORM_FILE into #list (with "do
                                  not close" option)
          execute processlist
     endwhile
     execute processlist

The above example will read any number of records from the input file, even though the list being used is efficiently sized with 100 entries. The list is acting like an input buffer for the application.

Some tips for using this option, and for using this Built-In Function, are:

• This function is designed to be an interface between Visual LANSA applications and external applications. It is designed to open a file, read data from it, then close it again. It is not designed to service more complex "system" level tasks such as maintaining an "always open" polling file.

• Up to 50 input files may be open concurrently. The operating system you are using may have limitations or configuration options that lower this limit.

• There is no limitation on maximum record length. At the end of every record a New Line character will be added as an End Of Record delimiter.

• When using the "Keep Open" option always let the Built-In Function continue until "EF" is returned. This means that the input file will have been closed.

• This Built-In Function must check all arguments every time it is called, and also search through a list of currently opened input files looking for a match. Therefore it is most efficient when called just a few times with lists allowing many entries, and least efficient when it is called many times with list(s) allowing just a few entries.

Record Selection Capabilities

Not available when the input file type is B.

Arguments 8, 9 and 10 allow simple record selection logic to be performed. If this option is enabled by passing argument 8 as a valid value other than N, then as each <record> is read from the file the following expression is evaluated:

if (substring (<record>,<position>,<length>) <operation> <value>)

where:

• <record> is the current record.

• <position> is the argument 9 position.

• <length> is the length of the argument 10 field. This <length> should not be bigger than 256 bytes.

• <operation> is the argument 8 operation.

• <value> is the value of the argument 10 field.

If the expression is found to be true, the record is selected. If the expression is false then the record is ignored.

Note that this comparison happens before any data processing.

It is a byte-orientated operation, not field-orientated operation.

If a file is planned to be used with Record Selection, it is recommended that you use a fixed length columns file ( C or D types in TRANSFORM_LIST ). It is also recommended that you move all the variable length fields toward the tail of the record in a such way so the <position > appears before those fields. The purpose of doing this is to make sure that the portion of <length> bytes, starting from <position> in the input buffer, has the same meaning for all records. Also if the <position> + <length > is greater than the number of bytes in the record, TRANSFORM_FILE will give this fatal error "Invalid arguments for comparison" and stop.

If the input file type is A, O or T, then any implied character appearing before the data to be compared should be counted in the <position > calculation.

Example:

An A type file is built from a list of 2 Alpha fields. Both are 9 bytes long. Fields of any entry are always full (occupy maximum length). There is no invalid character in the data. The comparison is intended to be from the first byte of the second field. So the <position> must be calculated like this:

     <position> = 1(opening double quote for the first field) + 9 (length of the first field) + (1 closing double quote) + 1 (coma ) + 1 (opening double quote for the second field) + 1 (<position> starts from 1 ) = 14

If the output file type is C or D the <position> calculation is:

       <position> = 9 (length of the first field) + 1 (<position> starts from 1 ) = 10

If you need help in understanding why a record is or is not located, turn tracing on to level 9, category 'BIF'. The trace file will list the comparison parameters used and the data that did not match the comparison parameters. Note that the trace is not in Unicode so Unicode data may not be displayed as you see it when using a Unicode-aware program like Notepad.

Appendage Lists

Up to 10 appendage working lists may be specified when invoking this Built-In Function. Appendage lists may be used when the input file contains more than 100 fields or where the aggregate entry length of a list exceeds 256 bytes.

Refer to the 9.219 TRANSFORM_LIST Built-In Function for more details of appendage lists. The concept of the "appendage option" field has no meaning to this Built-In Function. All fields defined in appendage list(s) are processed just like a logical extension of the primary list (argument 1). Appendage working lists should all have the same maximum number of entries allowed as the primary list.

Error Handling and Error Activity

The following table indicates the types of errors that you can trap at the RDML level with an "ER" return code (User Trap) and those that will cause a complete failure of your application (System Error). System errors invoke Visual LANSA full error handling and cause the entire X_RUN "session" to end. They cannot normally be trapped at the RDML level.

Error Handling Note

It is very strongly recommended that you avoid building complex error handling schemes into your applications. Use a very simple trap like this at all levels of your application.

     if (#retcode *ne OK)
        abort msgtxt('Failed to .............................')
     endif

Let the standard error handling Built-In Function to every generated application take care of the problem. Situations have arisen where user defined error handling logic has become so complex as to consume 40 - 50% of all RDML code (with no obvious benefit to the application). Do not fall into this trap.

For example:

To get a BLOB value BLOBNumber1.txt from a transformed file:

C:\Root\Data\Transformed1.dat
then the BLOB file itself must be:
C:\Root\Data\Transformed1_LOB\BLOBNumber1.txt

For a transformed file created NOT by TRANSFORM_LIST, BLOB (or CLOB) files need NOT to be duplicated. In this case the BLOB (or CLOB) value must have the full path in it.

For example, if you have a BLOB file called BLOBNumber1.txt currently located in C:\Data\ and if you do not  want to duplicate the BLOB file, the value of the BLOB field in the transformed file must be C:\Data\BLOBNumber1.txt.

Example

Refer to 9.219 TRANSFORM_LIST.