Integration fundamentals

Learning objectives

• Learn about the concepts and components used in integration including channels, messages, transformations, and services.

• Understand the key concepts of model services and the NexJ integration engine.

Key concepts

• Integration is the process of enabling different systems to communicate with each other. It attempts to address the challenges of enterprise environments, such as unreliable or slow networks, varied or incompatible applications, and changing framework needs.

• Enterprise environments often consist of many varied and complex systems. Most companies store information across a number of different systems, which can present challenges when information needs to be shared. Examples include:

  • Information is stored in a database that does not allow direct access.

  • A system requires synchronous communications that another system does not provide.

  • Information is stored as a file instead of in a database.

• NexJ approaches integration in the following ways:

  • File transfer - An application writes a file to a shared directory where it is read by another application.

  • Messaging - An application sends a message to a common message channel shared with other applications. Messaging is used for asynchronous communications, where the sender sends a message and moves on without requiring a response from the receiver.

  • Remote procedure call (RPC) - An application exposes functionality that can be accessed remotely by another application. RPC is used for synchronous communications, where the sender waits for a response from the receiver.

  • Shared database - Multiple applications share the same database schema in a single physical database. To carry out these approaches, the task of integration is broken up into a number of components. Each component has a role in the process of abstracting data, enabling it to be delivered between the applications and platforms being integrated.
    
    

• The integration engine is made up of the following six components:

  • Channels - Channels are what we send and receive messages over. They are abstract integration endpoints that represent the network transport layer, or protocol. Specific channel types include:

    • HTTP - both as a client and as a server to receive and send HTTP messages.

    • Message Queue - send and receive using JMS

    • UDP - send and receive using the User Datagram Protocol

    • TCP - send and receive using the Transmission Control Protocol

    • Mail - send and receive email messages

    • File - send and receive using a set of directories

    • Kafka - send messages to a Kafka Server

    • Generic - for a custom developed channel

    • Object Queue - domain model message queueing

  • Connections - Binds a channel to a physical endpoint.

  • Messages -A message carries information from a source system to a destination system over an integration channel. Inside the machine they are represented as Transfer Objects. On the wire, they take on different representations. This is called the wire format or the "serialized" message. To convert from wire format to internal format, you parse the message. To go the other way around, you format the message. Supported message formats include:

    • Object - Maps message parts to attributes in class instances

    • Fixed - Maps message parts to record fields in a fixed length file, or flat file database.

    • JSON - Maps message parts to JSON objects, arrays or primitives.

    • Avro - Maps message parts to Avro objects, arrays or primitives.

    • HL7 - Maps message parts to segments and fields in a Health Level Seven (HL7), v2.x file.

    • CSV - Maps message parts to columns in a CSV file.

    • XML - Maps message parts to XML elements, attributes or values. 

    • Zip - Maps message parts to files in a Zip file archive.

    • VCard - Maps message parts to elements in a vCard file.

  • Services- Services tie together integration structures and provide message orchestration. They are very similar to Workflows, but instead of a Class as the process token, a Service uses a message. Each step in a Service returns a message. Steps can:

    • run script,

    • send messages,

    • send and receive messages to external systems asynchronously,

    • transform messages

    • persist object messages to the domain model

    • sync object messages using the additional capabilities of the synchronization framework. The sync framework maintains a model of various sync targets and keeps track of each synchronized instance so it can detect conflicts. The sync step is really an extension of the persist step that allows for processing and conflict resolution.

    • try/catch blocks to handle exceptions

    • dispatch for branching based on conditions of the message or service

    • parse and format messages

  • Transformations -Transforms one type of message to another.

  • Interfaces - A list of request and response messages that are used to filter messages being sent and received.