Background: Creating a DataSource

Optional

This optional background material describes the steps we went through to create the new training:DB datasource used in the fundamentals training. You don't need to do this work, but it is provided so you can better understand the concepts of .datasource files, .upgrade files, and .environment files.

Creating a DataSource

To persist your data, it needs to be stored somewhere. Data sources define the structure of the data storage, not the physical implementation. You associate your data source with a physical system using environment file settings.

We are covering creating a .datasource file here so you get a good understanding of what this important concept is all about. That said, you will usually be using existing .datasource and .upgrade files and not have to create them that often.

Learning activity

This learning activities is not to be completed during training. It is for background information only and may be used as a guide if you need to create a new module.

There are three approaches to creating a data source, depending on your use case. An important concept with MODL development, is to avoid customization wherever possible. Augmentation is always better.

Use Case: Working within a project - If you are adding, or changing a class within a project, you may simply use that module's data source. For example, add your new tables to DefaultRelationalDatabase.datasource in finance.
Use Case: Enhancing a Project (with a namespace or mixin) - If you are enhancing the behavior of a project from another module, it is better to use a data source augment.
Use Case: Independent Module - If you are creating an independent module, you should create a new data source and optionally a new upgrade file. Create a new upgrade file if you are in a separate mixin project with a module name (namespace) set. The data source needs some initial settings, aspects, and a version table. This is shown in the following example.

Our training module is based on finance, but it really is independent. So we will be using folder namespacing and re-using the main.upgrade file. Let's define a new relational data source called training:DB.

In the Persistence >Data Sources layer, select New Data Source and complete the information as follows.
Click Finish.
Press the Save button.

Important

This is important so you have a starting point in the source code control's local history. We will use this to generate upgrade steps going forward.

Paste the following into the Source tab of your data source file.

XML

<RelationalDatabase aspects="training:CORE_ASPECTS" prefix="TRN" versionTable="Version">
   <Tables>
      <Table aspects="!LOCKING !PARTITIONED" name="Version" primaryKey="Version.PK" type="managed">
         <Columns>
            <Column allocation="varying" caseInsensitive="false" description="Metadata namespace" name="namespace" nullable="false" precision="128" type="string"/>
            <Column allocation="varying" caseInsensitive="false" description="Metadata version" name="version" nullable="false" precision="128" type="string"/>
            <Column description="Metadata upgrade step within the version, -1 if the version is up to date" name="step" nullable="false" type="integer"/>
            <Column description="True if the schema may be upgraded automatically" name="upgradable" nullable="false" type="boolean"/>
            <Column description="True if the schema contains test data, which can be recreated automatically" name="test" nullable="false" type="boolean"/>
            <Column description="True if the initial (seed) data has been loaded" name="loaded" nullable="false" type="boolean"/>
            <Column caseInsensitive="false" description="Prefix for data encrypted with the activeCipherKey" name="activeCipherPrefix" type="integer"/>
            <Column caseInsensitive="false" description="The public key of the active cipher PKI key pair" name="activeCipherKey" type="binary"/>
            <Column caseInsensitive="false" description="The public key of the obsolete cipher PKI key pair" name="obsoleteCipherKey" type="binary"/>
         </Columns>
         <Indexes>
            <Index name="Version.PK" type="cluster" unique="true">
               <IndexColumn name="namespace"/>
            </Index>
         </Indexes>
      </Table>
   </Tables>
</RelationalDatabase>

Press Save. Go back to the Schema tab of the Relational Datasource Editor for your new training:DB data source and you should see something like the following image on the Columns tab.

Relational Datasources

A relational datasource is a logical representation of the structure of a database. You can read more about them in the Model Definition Language Reference.

We want to be able to track changes between versions of the database for our new datasource, so we include a Version table. This is a standard table that keeps track of the current state of the database. You can look at what columns and indexes it has in the editor. During these lessons, we will be adding more tables to this datasource to support persisting our classes.

As mentioned earlier, in an independent mixin project with a module name set, we would need to create a .upgrade file, but we are using the inherited main.upgrade from finance in our training.

Editing the upgrade file

To persist your data, you need to update the physical databases from our logical datasource to reflect the changes.

To do this, you'll need to declare the database changes in the upgrade file. An upgrade file contains one or more upgrades, each associated with a specific version of the model. Each upgrade contains steps that define the changes that take place from an old version of the model to the new one associated with the upgrade. Once an upgrade file has been defined for the current version of a model, you can use it to migrate the data store for any previous version of the model to the current one.

NexJ Studio can create the upgrade steps by comparing the current version of the model to an earlier version of the model. In this case, compare your changes (that is, the addition of the Version table) to a published model or local history.

Upgrade Files

An upgrade file is set of changes to a Relational Datasource from version to version. It is used by tools like the Data Load Tool and the Database Schema Tool to bring databases up to date in different environments. You can read more about them in the Model Definition Language Reference.

Learning activity

This learning activities is not to be completed during training. It is for background information only and may be used as a guide if you need to create a new module.

To update the upgrade file:

In the Persistence layer, click the Data Sources tab.
Right-click click on the training:DB.datasource and select Generate Upgrade Steps...
Set the Current Data Source to training:DB and the Upgrade should be Main.
Under Old Data Source, select Local History.

The training version of studio is configured to keep old revisions around, even for large files. This is controlled with the Window/Preferences... General/Workspace/Local History options.

Select the first revision time entry
This allows you to look at the changes you've made to the file recently and find the appropriate old version to start from.

Another approach would be to use Old Model and use a previously published original model as the old model.
You want to select the Revision that doesn't have your table in it. You can look at the differences by double clicking on the Revision History items. Your compare should look something like:
Press Next. Change the version to 2.1092.1307.167.53, set the description to "Added the training:DB Version table" and save.
You always must ensure that your current model's version matches the upgrade version. Click the Set current model button
in the toolbar to open the Model Library.
With the training model selected, click Edit. Update the Model Version field to match the version associated with the upgrade you just created. If your upgrade is associated with version 2.1092.1307.167.53, change the Model version to 2.1092.1307.167.53. This enables NexJ Studio to recognize that the model has changed from the previous version and that your update needs to be applied to the supporting data stores.
Click Apply and Close. The new version is now reflected in the table in the Models tab.
Click Close.
We need to make our Environment file aware of our new datasource. Click the set current server button
and navigate to the Data Source Connections bottom tab. Select the Relational Database Connection and press the arrow in the lower Data Sources list.
Add training:DB to the selected data sources and press OK, then Save.
Select Run/Run Tool/Data Load Tool from the menus and run the recreate command against the training:DB. This will establish a starting point for your TRNVersion table.
Then use the same tool to run the upgrade command on the DefaultRelationalDatabase datasource.

For locking you work correctly, you need to create a new training:LOCKING (meta/training/LOCKING.meta) aspect with the following contents.

training:LOCKING aspect

XML

<Aspect>
   <Attributes>
      <Attribute initializer="0" name="locking" required="true" type="integer"/>
   </Attributes>
   <PersistenceMappings>
      <PersistenceMapping dataSource="training:DB" lockingAttribute="locking">
         <RelationalMapping primaryTable="LOCKING">
            <AttributeMappings>
               <AttributeMapping column="locking" name="locking"/>
            </AttributeMappings>
         </RelationalMapping>
      </PersistenceMapping>
   </PersistenceMappings>
</Aspect>

This background material showed you how we set up our new data source. As stated earlier, you don't need to do this often, but it's good to understand the concepts.