Skip to main content
Skip table of contents

Lesson - Fixed Width Messages

In this lesson, you continue the exploration of the Integration Engine. You will receive a fixed-width file of entity records over a file channel and process the received information by logging it to the server console.

On completing this lesson, participants will:

  • have gained experience working with the Fixed message format, file channels and integration services
  • be able to describe the purpose of the fixed-width message format

Key Concepts

In this lesson you are introduced to the Fixed message format.

Fixed Width Messages

A fixed-width file (or flat file database) is a text file where the number and widths of fields is fixed. Data in each field is padded using a padding character (default is the SPACE (\u0020) character). Fixed files have no delimiters between records and fields and do not require newlines between records, so they are often simply a single line of text unless the data itself contains newline characters.

A valid Fixed message must have the following structure:

CODE 
Root-+ 
     | records-+ 
               | 
               + field_1 
               + field_2 
               + ...

where Root and records are Message parts and field_N are Value parts. All three levels may have Fixed mappings, allowing the setting of extra attributes. The Root mapping only supports a Prefix. The records mapping supports Prefix, Suffix and Paging. The field_N mapping supports Prefix, Suffix, Padding, Alignment, Format and the mandatory Width.

Width is only specified on the Value (field) parts mapping and is required. The width value must include the length of the prefix and suffix for that field. The contents of the field will be the field's prefix and suffix (if any), its data, and padding characters to fill out the remainder of the field width.

Record and field mappings may have Prefix and/or Suffix attributes set. The Root part may only have a Prefix (i.e. the start of the file). During parsing of messages, the Fixed parser simply skips over the prefix and suffix parts, it does not do string comparisons of this data. During formatting, the provided prefix/suffix values are written out. Typically, fixed-width files do not use prefixes or suffixes.

The Padding attribute denotes the single character that is used to fill out the remainder of the field width beyond the provided data. During parsing, this character is trimmed from the front and back of the field's data; it is not removed if it is between non-padding character data. During formatting the padding character is added on one side of the data based on the field's Alignment attribute.

Alignment is only specified on the Value (field) parts mapping and defaults to left. Alignment indicates to which side of the field the data is to be aligned, meaning that padding characters will be added to the opposite side of the data (e.g. left alignment means padding characters are added to the right side of the data).

Format is a string to denote how the field data is to be displayed during formatting. For example, the format attribute for a timestamp value is used by Java's SimpleDateFormat when outputting the value as a string.

The Fixed message adapter will use paging if the paging attribute is set on the records part, meaning that the adapter can parse extremely large amounts of text from a single file. The value specified is the number of records that are parsed into memory before writing those records to a temporary paging file on disk. This allows for the processing of extremely large numbers of records in a single flat file. The paging file is created in the temporary directory of the environment where the system is running. As per the documentation for java.io.File.createTempFile(java.lang.String, java.lang.String, java.io.File):

The default temporary-file directory is specified by the system property java.io.tmpdir. On UNIX systems the default value of this property is typically /tmp or /var/tmp; on Microsoft Windows systems it is typically C:\\WINDOWS\\TEMP. A different value may be given to this system property when the Java virtual machine is invoked.

If using paging, you must ensure that the temporary directory has enough free disk space to store the paged data of all concurrently processed paged messages.

For more information on fixed-width messages see the Model Description Language Reference.

Fixed Message Import

In this exercise, we'll receive a fixed-width text file on a file channel and log its contents to the server's console.

Step 1: Create the Message

To create the message that the Service will consume:

  1. In the Integration > Messages tab, create a new message called WidgetsFixed.
  2. Select the root part of the message and set it's format to Fixed.
  3. Click the Create Mapping button and keep the mapping defaults.
  4. Either from the Outline view or in the message editor, right-click the root node of the message and select Insert Child > Message.
  5. In the Properties view for this new message node, set:
    • Name: records
    • minCount: 0
    • maxCount: 0
      Note: A maxCount of 0 means a collection of unlimited size.
  6. Under the records node, add the following child values and associated mapping values:
    • Name: name, Type: string
      • Width: 20
    • Name: count, Type: integer
      • Width: 10
    • Name: lastCounted, Type: timestamp
      • Width: 40, Format: yyyyy.MMMMM.dd GGG hh:mm:ss aaa

Step 2: Format and Parse the Message

Before you start working with services and channels in this example, you'll look at formatting and parsing in action.

Start your Scheme Console and run the following code from a scratchpad.

SCHEME 
(define om ; original manually created message 
   (message 
      (: :class "WidgetsFixed") 
      (: records 
         (collection 
            (message (: name "prybar") (: count 42) (: lastCounted (now))) 
            (message (: name "hammer") (: count 101) (: lastCounted
 #m1999-12-31T23:59:59.999))
            (message (: name "paint brush") (: count 16) (: lastCounted
 #m2010-02-03T11:22:33.444))
         ) 
      ) 
   ) 
) 
(define fm ; formatted message 
   (format-message om) 
) 
(define pm ; parsed message 
   (parse-message fm "WidgetsFixed") 
)

This script manually creates an internal WidgetsFixed message, formats it to its wire format (Fixed) and then parses that back to internal format. After you run this code, just type fm in the console to see the formatted message.

Step 3: Create the Service, Channel and Interface

  1. Create a service in the Integration > Services tab called WidgetsImportService.
  2. In the Integration > Channels tab, create a new File channel called WidgetsImportChannel. This is done by using the New Channel Wizard, entering the name, and selecting File as the channel type.
  3. Enter a description for the channel and keep all of the other default settings.
  4. In the channel's Service Bindings tab, add a new service binding for the WidgetsImportService. Leave the Output property empty.
  5. Create a new Interface in the Integration > Interfaces tab called WidgetsFixedInterface. Leave its format blank and add the WidgetsFixed message to the Requests list.
  6. In the WidgetsImportService, set the service's interface property to the WidgetsImportInterface. So you have:
    • an incoming message type (WidgetsFixed),
    • the channel that it will be received on (WidgetsImportChannel),
    • a binding from this channel to the service that will process the message (WidgetsImportService),
    • and an interface on that service that will be used to automatically validate and parse the incoming message (WidgetsImportInterface).

  7. Add a log step to the service and set its Arguments property to:

    CODE 
    "Incoming records:" (this'records)

Step 4: Add the Channel to the Environment

Connect the abstract channel to the physical world. You do this by configuring a connection in your environment:

  1. Open the Development environment file.
    Note: A shortcut to open your current environment is clicking the Set current server 
     toolbar button.
  2. In the Channel Connections tab, add the CSVImportChannel channel.
  3. Select the new channel and set the following properties.:
    • Incoming Directory: c:\temp\file\in
    • Processed Directory: c:\temp\file\processed
    • Temporary Directory: c:\temp\file\temp
    • Interval: 5000
    • Incoming Message File Age: 5000
    • Outgoing Directory: c:\temp\file\out
    • Journal Directory: c:\temp\file\journal

You may substitute paths more appropriate to you system.


The directories specified in the file connection need to exist. If they do not, create them now.

Step 5: Test the Basics

  1. Create a file called widgets.txt and enter the following text:

    CODE 
    screwdriver 223 02010.April.06 AD 04:44:38 PM
      monkey wrench 7 2009.December.21 AD 11:22:33 AM

    Note: The format of this text file is important:

    • All of the text should exist on a single line.
    • The number of characters in a field must match the field's definition in the message. Ensure that the appropriate number of padding characters (spaces) are added to each field.
  2. Stop any Scheme consoles you may have running currently.
  3. Run the Scheme Server Console to start a fresh server.
  4. When the server has launched, copy the file widgets.txt to your Incoming Directory as specified in the Development environment.

You will see activity in your server log. The file will be removed from file\in directory and a copy will appear in the file\processed folder with a new date-modified filename.

Step 6: Housekeeping

Since you did not change the database schema in this lesson, you do not edit the Upgrade file, upgrade the UnitTest dump file or upgrade the database.


JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.

Contact Support Close