Synchronization Engine Concurrency

We have implemented some changes to better manage concurrency in bi-directional synchronization scenarios. The biggest change will be to permit outbound messages to be grouped by target or link. Each group of messages will be processed on its own nested transaction. At the beginning of each nested transaction, a hook on the target will be called: this will permit the target to take precautions against conflicts from concurrently running processes. In the case of Exchange sync, we will use the hook to acquire a lock on the ExchangeFolder (SysSyncLink).

Here are the Exchange sequence diagrams.

Exchange Inbound Sync:

When inbound synchronization is initiated, synchronization meta-information (SyncTarget, SyncFolder, etc.) is loaded from the database. The ExchangeFolderLock instance associated with the folder being synchronized is locked, to prevent database updates from concurrently running inbound synchronization processes on the same folder, and to avoid reading partial changes generated by concurrently running outbound synchronization processes. Inbound changes are then read from Exchange using one or more service requests. The bulk of the database updates are performed in one or several nested transaction, after which some additional updates to TaskSeries and to SyncObjects may be performed.

Exchange Outbound Sync:

Outbound synchronization is initiated when a transaction with changes to synchronized instances commits. The originating transaction posts, through JMS, one or more SyncCommands to send updates to the external system (these request may be generated asynchronously on one or more additional transactions, not shown.) The originating transaction commits the changed instances to the database before a SyncCommand is invoked.

At some later time, the SyncCommand is read through queues. All instances specified by the originating transaction, along with associated meta-information (syncObjects, syncGroups, syncLinks, etc.) are reloaded from the database. The instances are parsed into ObjectMessages; if any instances change during the parsing process the SyncCommand fails and is later retried. Messages are grouped by folder: for each folder, a separate transaction is run.

The first action on each nested transaction is to lock the ExchangeFolderLock associated with the folder, to prevent Exchange update operations from conflicting with other synchronization processes (inbound or outbound). The second action is to read the SyncObjects table, to verify that SyncObjects about to be created have not already been created by another outbound process. The updates (derived from the ObjectMessages) are sent to Exchange in one or several service requests. Interleaved with these service requests, reads may be done on synchronization meta-information that was not previously identified as relevant by the Framework. All involved syncObjects and syncGroups are updated with the identifiers and versions returned by Exchange. Finally a transaction listener fires to move any draft invitations to the outbox in Exchange, causing them to be sent. In the event of rollback, a transaction listener will remove any freshly created items: this ensures that we do not create items in Exchange for which we have no corresponding SyncObjects.

The "transaction listener" mentioned above is specifically designed for outbound synchronization with non-transactional targets. It allows some operations to be deferred until after our unit of work commits changes to the database. A call to nexj.core.runtime.FunctionSynchronization'submit registers a listener that will be called whenever the current transaction is committed or rolled back.