Skip to main content
Skip table of contents

File integration



Learning objective
In this lesson, you will learn to:

  • Create a CSV file that contains the contact details of some people.
  • Write a script to parse and format the details to a message.
  • Persist the message to the business model through a channel.
  • Investigate how a service interacts with the file system. 
  • Configure a channel to use a service and interface to process and parse the message.

Creating a CSV message

Let's create a new message called training:PersonCSV that stores the first name, last name, business phone number, home phone number, and email address of a person. The person's details are stored in a child message called rows. A default mapping is applied to the training:PersonCSV message. The default mapping specifies that the file has a header row and sets other default options. It looks similar to the following:

Example default mapping

Example default mapping

CVS message mapping options are described in detail in the Model Description Language Reference at CSVMapping (nexj.com)

Learning activity

To create the CSV message:

  1. In the Integration layer, select the Messages tab.

  2. Right-click the list area inside the tab and select New Message. The New Message dialog opens.

  3. Name the message training:PersonCSV.

  4. Click Finish.

  5. Do one of the following:

    • Enter or drag and drop the following code on the Source tab.

      XML 
      <Message format="CSV">
   <CSVMapping/>
   <Parts>
      <Message maxCount="0" name="rows">
         <Parts>
            <Value name="firstName" type="string"/>
            <Value name="lastName" type="string"/>
            <Value name="businessPhone" type="string"/>
            <Value name="homePhone" type="string"/>
            <Value name="email" type="string"/>
         </Parts>
      </Message>
   </Parts>
</Message>

    • Configure the following settings:

      1. In the Overview tab, select the training:PersonCSV root node.

      2. Set the Format to CSV.

      3. Click Create Mapping. Do not change any of the CSV Mapping default values.

      4. Right-click the training:PersonCSV root node, select Insert Child > Message, and specify the following details:

        • Name: rows

        • maxCount: 0

      5. Right-click the rows node and select Insert Child > Value for each of the fields listed below. For each of these fields, set the name property and set the Type to String.

        • firstName

        • lastName

        • businessPhone

        • homePhone

        • email

10. Save your work.

Formatting and parsing the message

Using a scratchpad and Scheme script, you will create an unformatted message, format it, and then parse it into a defined message format. The first statement in the script creates two messages in the rows collection and specifies values for the fields. The second statement converts the message into its wire format, a CSV string. The third statement parses the CSV string into the specified message format.

Learning activity

To format and parse the message:

  1. Enter the following code in a scratchpad.

    SCHEME 
    ; Unformatted original message
(define om
	(message
		(: :class "training:PersonCSV")
		(: rows
			(collection
				(message
					(: firstName "Joe")
					(: lastName "Test")
					(: homePhone "(416) 555-1212")
				)
				(message
					(: firstName "Jane")
					(: lastName "Doe")
					(: email "jane.doe@nexj.com")
				)
			)
		)
	)
)
; Formatted message
(define fm
	(format-message om)
)
; Parsed message
(define pm
	(parse-message fm "training:PersonCSV")
)

  2. Run your server.

  3. In the Console, execute each of the statements from the scratchpad individually and review the results.

  4. Execute the statement (display fm) to see the formatted message, and (display pm) to see the parsed message.

Creating a service, channel, and interface for the message

You now need to create a channel that receives the incoming training:PersonCSV message. The channel is bound to a service that processes the message using an interface that validates and parses it.

Learning activity

To create a new service called training:CSVImportService:

  1. In the Integration layer, select the Services tab.

  2. Right-click the list area inside the tab and select New Service.

  3. In the New Service dialog, name the service training:CSVImportService.

  4. Click Finish. The new service you created opens in an editor.

To create a new channel called training:CSVImportChannel and bind the training:CSVImportService to the channel:

  1. In the Channels tab, right-click the list area inside the tab and select New Channel. 

  2. In the New Channel dialog, specify the following values:

    • Name: training:CSVImportChannel

    • Type: File

  3. Click Finish. The channel opens in an editor.

  4. Do one of the following:

    • Enter or drag and drop the following XML on the Source tab:

      XML 
      <File>
   <ServiceBindings>
      <ServiceBinding service="training:CSVImportService"/>
   </ServiceBindings>
</File>

    • Configure the following settings:

      1. In the Service Bindings tab, click the Add button Add button. The new service binding is added to the list in the Service Bindings area.

      2. Set training:CSVImportService as the Service.

      3. Leave the Output property blank.

  5. Save your work.

To create a new interface called training:PersonCSVInterface, format the interface as CSV, and set the training:PersonCSV message as a request type:

  1. In the Interfaces tab, right-click the list area inside the tab and select New Interface.

  2. In the dialog, name the interface training:PersonCSVInterface.

  3. Click Finish. The interface opens in an editor.

  4. Do one of the following:

    • Enter or drag and drop the following XML on the Source tab.

      XML 
      <Interface format="CSV">
   <Requests>
      <Request message="training:PersonCSV"/>
   </Requests>
</Interface>

    • Configure the following settings:

      1. In the Format field, select CSV.

      2. In the Requests area, click the Select button , add the training:PersonCSV message, and close the Select Messages dialog.

  5. Save your work.

To set the interface and add a log to the service:

  1. Open training:CSVImportService.

  2. Do one of the following:

    • Enter the XML code as follows, or drag and drop the XML on the Source tab.

      XML 
      <Service interface="training:PersonCSVInterface" layout="startX:133;startY:176;endX:400;endY:173">
   <Log args="(this'rows)" id="&quot;Incoming rows: {0}&quot;" layout="x:240;y:176" level="&quot;info&quot;" name="log"/>
</Service>

    • Configure the following settings:

      1. In the Property Editor view, set the Interface to training:PersonCSVInterface.

      2. In the Palette, select Log and click a location on the line between the start and end of the service to place the log.

      3. In the Property Editor view for the log, select the Parameters tab.

      4. In the Level field, enter "info".

      5. In the Id field, enter "Incoming rows: {0}"

      6. In the Arguments field, enter (this'rows).

  3. Save your work.

Adding the channel to the development environment

Connect the channel to the physical world by configuring a connection in your environment.

Learning activity

To add the channel to the development environment:

  1. In the Deployment layer, select the Environments tab and double-click the Development environment to open it for editing.

  2. In the Channel Connections tab, click the Select Channel Connections button Select Channel Connections button and add training:CSVImportChannel.

  3. In the Channel Connections area, select the new channel and set the following properties. If necessary, substitute paths that are more appropriate to your system.

    • Incoming Directory: C:\temp\CSVImport\in

    • Processed Directory: C:\temp\CSVImport\processed

    • Temporary Directory: C:\temp\CSVImport\temp

    • Outgoing Directory: C:\temp\CSVImport\out

    • Interval: 5000

    • Incoming Message File Age: 5000

      As an alternative to setting your properties individually, you can paste the following code into your environment file:

      XML 
      <FileConnection age="5000" channel="training:CSVImportChannel" incomingDirectory="C:\temp\CSVImport\in" interval="5000" outgoingDirectory="C:\temp\CSVImport\out" processedDirectory="C:\temp\CSVImport\processed" temporaryDirectory="C:\temp\CSVImport\temp"/>

  4. In your computer's file system, create the corresponding folders. This is required because the folders specified in the channel connection must exist on your computer in order for the channel to operate correctly. 

    • C:\temp\CSVImport\in

    • C:\temp\CSVImport\processed

    • C:\temp\CSVImport\temp

    • C:\temp\CSVImport\out

  5. Save your work.

Testing the message processing

In this learning activity, you will test the message processing.

Learning activity

To test the message processing:

  1. In your computer's file system, create a new file called test.csv and add the following text into it. Do not create this file in any of the folders you specified for the channel connection.

    TEXT 
    firstName,lastName,businessPhone,homePhone,email
Joe,Test,,(416) 555-1212
Jane,Doe,,,jane.doe@nexj.com


  2. Run the Server Console.

  3. Once the Console has launched, copy the test.csv file to the Incoming Directory that you created in your computer's file system.
    In the Console view, you should see processing information similar to the following:

    TEXT 
    Incoming rows: #<[TO<, @1762367942>(
	lastName="Test",
	firstName="Joe",
	homePhone="(416) 555-1212"
), TO<, @2023541770>(
	lastName="Doe",
	firstName="Jane",
	email="jane.doe@nexj.com"
)]>


  4. In your computer's file system, inspect the CSVImport folder and its subdirectories. The test.csv file was moved from the in folder to the processed folder, and the file now has a time stamp appended to its name.

  5. Stop the Scheme Console by clicking the Terminate button

    .

Persisting the message

Before persisting the data, you'll need to add some script steps to the service to persist the Person and Telcom information through the business model.

Learning activity

First, you'll need to add a user to the channel so you can do some data work. If the channel does not have a user associated with it, the message will be received and processed but the service fails when creating the Person instances.

  1. Open your environment file.

  2. In the Channels tab, open training:CSVImportChannel.

  3. In the Overview tab, set the Default User to nexjsa.

  4. Click the Save button Save button  in the toolbar.

To persist the data:

  1. In the training:CSVImportService, add a script step to the service diagram. Place it after the log step.

  2. In the General tab of the Property Editor for the script step, set Name to createPersons and Caption to Create person records.

  3. In the Script tab, enter the following code:

    SCHEME 
    (logger'debug "=====> this: " this)
(logger'debug "=====> rows: " (this'rows))

(for-each
	(lambda (row)
		(logger'debug "=====> row: " row)
		(let ((person
			(Person'new
				(: lastName (row'lastName))
					(: firstName (row'firstName))))
				)
				(when (not (null? (row'email)))
					(EmailAddress'new
						(: name "Email")
						(: address (row'email))
						(: type (read-instance TelcomType '() '(= name "Email") '()))
						(: entity person)
					)
				)
				(when (not (null? (row'homePhone)))
					(TelephoneNumber'new
						(: name "Home")
						(: address (row'homePhone))
						(: type (read-instance TelcomType '() '(= name "Home") '()))
						(: entity person)
					)
				)
		)
	)
	(this'rows)
)


  4.  Save your work.
    The service diagram should display as follows:

    Service diagram

    Service diagram


To test the training:CSVImportService:

  1. Run the Server Console.

  2. After the Console launches, copy the test.csv file to the Incoming Directory.
    You should see activity in the Console view. The file is taken from the in folder and output to the processed folder. The file now has a date-modified string appended to the file name.

  3. Inspect the Console output to see how the framework persisted the Person and Telcom objects.

  4. Stop the Server Console.

You can use the user interface at http://localhost:7080/training/ui/portal to look up contacts with the last names of Test or Doe. 

Alternately, you can use the REST API with:
http://localhost:7080/training/json/Person?attributes=(firstName lastName)&where=(or (= lastName "Test") (= lastName "Doe"))

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.

Contact Support Close