Activating the Persistence Service 7

The Persistence Service provides a persistent key-value store to Transformer modules and StreamLink clients. The service can be configured to persist data to a database server or to a local SQLite file.

This page is for the Persistence Service included with Transformer 7.0. For the Persistence Service included with Transformer 6.2, see Activating the Persistence Service (Transformer 6.2).

Contents:

Dependent modules

The following Caplin-supplied Transformer modules are dependent on the Persistence Service:

  • Persistence Service Client
  • Watchlist Service
  • Alerts Service
  • High-Low Service

Upgrading from the Persistence Service 6.2

The Persistence Service 7.0 has a new API that is not compatible with modules written for the Persistence Service 6.2 API.

Custom modules written for the Persistence Service 6.2 API must be upgraded to the Persistence Service 6.2 API.

For a summary of breaking changes, see Upgrading to Caplin Platform 7: Persistence Service.

Activating the Persistence Service blades

The Persistence Service comprises two blades, which are deactivated by default:

  • PersistenceService: This blade provides persistence services to Transformer modules, including the PersistenceServiceClient module (below).
  • PersistenceServiceClient: This blade provides persistence services to StreamLink clients. It requires the PersistenceService module to be active.

To activate the Persistence Service blades, run the commands below from the root of your Deployment Framework:

$ ./dfw activate PersistenceService
$ ./dfw activate PersistenceServiceClient

Persistence stores

The Persistence Service supports the following persistance stores:

File-backed persistence

Data is persisted to database tables in a local SQLite file. File-backed persistence is configured by default.

File-backed persistence cannot be used in a Transformer 7 cluster. Transformer 7, unlike Transformer 6.2, does not synchronise persisted data between cluster nodes.

Warning: file-backed persistence is unsupported in production environments, and, from version 7 of the Persistence Service, file-backed persistence cannot be used in Transformer clusters.

When Transformer is started with file-backed persistence enabled, the Persistence Service performs the following operations:

  1. Creates the SQLite database file specified by add-database-params/init-string (if the file does not exist).
  2. Creates the database table required by the now deprecated JTM API, com.caplin.transformer.module.persistence (if the table does not exist).
    1. The filename of the SQLite DDL script is specified by the persistence-filedb-bootstrap configuration option in the file <Framework-root>/active_blades/PersistenceService/Transformer/etc/persistence.conf
  3. Creates the database tables required by each Transformer module that uses the Persistence Service 7's API (if the tables do not exist).
    1. The filename of each module's SQLite DDL script is specified by the persistence-filedb-bootstrap configuration option in the module's file Transformer/etc/persistence-extra.conf

Database-backed persistence

Data is persisted to database tables in a relational database server.

Database-backed persistence is the only form of persistence storage that is suitable for use in a Transformer 7 cluster, and the only form of persistence supported in production.

The Persistence Service supports the following database servers:

  • Oracle
  • Microsoft SQL Server
  • Sybase (tested with the Sybase jConnect for JDBC driver)
  • MySQL (tested with Oracle Connector/J for MySQL)
  • Postgres
  • H2

Unlike file-backed persistence, database-backed persistence does not include bootstrap routines to create database tables in your chosen database. You must create the tables required by dependent Transformer modules.

Configuring file-backed persistence

File-backed persistence is configured by default.

Support for SQLite is built-in to the Persistence Service. No external driver is required.

To revert to file-backed persistence after changing the configuration, edit the file <Framework-root>/global_config/overrides/PersistenceService/Transformer/etc/persistence.conf so that it is consistent with the values in the table below.

Configuration item Value Comment
enable-file-database TRUE Set to TRUE to enable file-backed persistence.
add-database-params/init-string ${ccd}/persistence.db The filename for the SQLite database. If the database does not exist, the Persistence Service will create it. For a list of valid filename formats, see URI Filenames in SQLite C Interface: Opening a new database.

Configuring database-backed persistence

To configure the Persistence Service to use database-backed persistence:

  1. Create database tables and triggers
  2. Add the database's JDBC driver to Transformer's classpath
  3. Configure the connection to the database

1. Create database tables and triggers

Use a database administrator account to create the tables and triggers used by .

For Caplin-supplied modules, adapt the SQLite scripts in the table below to the syntax of your database server:

  SQLite bootstrap script Notes
Persistence Service 7 /active_blades/PersistenceService/Transformer/etc/bootstrap.sql Creates the table required by the Persistence API in the deprecated JTM, com.caplin.transformer.module.persistence.
Persistence Service Client 7 /active_blades/PersistenceServiceClient/Transformer/etc/bootstrap.sql  
Watchlist Service 7 /active_blades/WatchlistService/Transformer/etc/bootstrap.sql  
Alerts Service 7 /active_blades/AlertsService/Transformer/etc/bootstrap.sql  
High-Low Service 6.2 - Version 6.2 uses the Persistence API in the deprecated JTM, com.caplin.transformer.module, and so does not have its own persistence table.

2. Add the database's JDBC driver to Transformer's classpath

To add the database's JDBC driver to Transformer's classpath:

  1. Create the following directory: <Framework-root>/global_config/overrides/PersistenceService/Transformer/lib/java
  2. Copy the JDBC driver JAR file to the directory created in step 1.
  3. Make the following two changes to the file: <Framework-root>/global_config/overrides/PersistenceService/Transformer/etc/java.conf
    1. Add the following line (where jar_filename is the name of JDBC driver JAR file):
      1. jvm-global-classpath "${ccd}/../lib/java/jar_filename"
    2. Uncomment the following line:
      1. jvm-location ${JVM_LOCATION}

3. Configure the connection to the database

To configure database-backed persistence, edit the file <Framework-root>/global_config/overrides/PersistenceService/Transformer/etc/persistence.conf so that it is consistent with the values in the table below.

Configuration item Value Comment
enable-file-database FALSE Set to FALSE to use database-backed persistence.
persistence-classid com/​caplin/​transformer/​persistence/​JdbcPersistorImpl The implementation of the Persistor interface uses to persist data to the database. The example value in this table is the default JDBC Persistor that comes packaged with the Persistence Service.
add-database-params/init-string jdbc:mysql:/​/​localhost/​rtas The JDBC URL of the database. The example value in this table is for a local MySQL database called 'rtas'.
add-database-params/driver-name com.mysql.jdbc.Driver The name of the JDBC driver's main class. The example value in this table is for the Oracle MySQL Connector/J driver.
add-database-params/username user1 The username of the Persistence Service's database account.
add-database-params/password password The password of the Persistence Service's database account.
add-database-params/extra-params 5 Extra parameters accepted by the Persistor. The com.caplin.transformer.persistence.JdbcPersistorImpl accepts one parameter: the maximum time to wait, in seconds, for the operation used to check the validity of a database connection to complete. See Connection.isValid. Default value 5.

Configuring permissioning

To allow users to interact with the Persistence Service Client, configure Liberator's Permissioning Service according to the specification below. For more information about permissioning, see User Authentication and Permissioning.

Rules

Define rules that trigger when users contribute (publish) to the Persistence Service Client's CONTROL and RECORD subjects. The strings used for the permission namespace and for the action are arbitrary; you are free to customise them to your own naming scheme.

Message Subject Rule Type Permission Namespace Action Action Ref Product Ref
^/PRIVATE/PERSISTENCE/%u/CONTROL WRITE PERSISTENCE_SERVICE PERSISTENCE-CONTROL - ALL_PRODUCTS
^/PRIVATE/PERSISTENCE/%u/RECORD/.* WRITE PERSISTENCE_SERVICE PERSISTENCE-RECORD - ALL_PRODUCTS

Permissions

Assign users the following permissions.

Product Set Permission Namespace Action Authorisation
.* PERSISTENCE_SERVICE PERSISTENCE-CONTROL ALLOW
.* PERSISTENCE_SERVICE PERSISTENCE-RECORD ALLOW
^/PRIVATE/PERSISTENCE/%u/CONTROL - VIEW ALLOW
^/PRIVATE/PERSISTENCE/%u/RECORD/.* - VIEW ALLOW

See also: