Activating the Persistence Service

The Persistence Service provides Transformer modules and StreamLink clients with a key-value store, persistent across server restarts and shared between nodes in a Transformer cluster.

The Persistence Service comprises two built-in blades that are included with Transformer from version 6.2.3. Neither blade is active by default.

The Persistence Service is required by the following Transformer services: Watchlist, Charting, and High-Low.

Activating the blades

There are two Persistence Service modules:

  • PersistenceService: This module provides persistence services to Transformer modules, including the PersistenceServerClient module. See the Java package com.caplin.transformer.persistence.

  • PersistenceServiceClient: This module provides persistence services to StreamLink clients. It requires the PersistenceService module to be active.

To activate the Persistence Service blades, run the commands below from the root of your Deployment Framework:

$ ./dfw activate PersistenceService
$ ./dfw activate PersistenceServiceClient

Configuring the Persistence Service

Follow the instructions in this section to configure the Persistence Service.

Deciding which datastore to use

Transformer’s Persistence Service supports both local and shared datastores.

Local datastore (filesystem)

Data is persisted to Transformer’s local filesystem. This is the default configuration and is suitable for development use only.

File-backed persistence is unsupported in production environments, and, from version 7 of the Persistence Service, file-backed persistence cannot be used in Transformer clusters.

Shared datastore (database)

Data is peristed to a SQL database server. This configuration is supported for use in production use.

This datastore can be shared between instances of Transformer and so is suitable for clustered deployments.

To configure database-backed persistence:

  1. Create the database table to store persistence data.

  2. Add the database’s JDBC driver to Transformer’s classpath.

  3. Configure the Persistence Service, providing connection details to the database.

Configuring a shared datastore

Create the database table

Choose a SQL database that supports JDBC access over the network using text-based authentication credentials.

Create a table with two text columns, one to store persistence keys and the other to store persistence values. Define the column that stores the persistence keys as the primary key of the table. You may choose any name for the table and its columns: the table name and column names are configurable in the Persistence Service’s configuration file (see Defining the persistence configuration items).

Here’s an example MySQL table definition that uses the table and column names defined in the Persistence Service’s default configuration file:

CREATE TABLE persistence (
    pers_key VARCHAR(100),
    pers_val TEXT,
    PRIMARY KEY (pers_key)
);

You may need to increase the width of the 'pers_key' column if a service or client uses keys greater than 100 characters in length.

Add the database’s JDBC driver to Transformer’s classpath

To add the database’s JDBC driver to Transformer’s classpath:

  1. Create the following directory: <Framework-root>/global_config/overrides/PersistenceService/Transformer/lib/java

  2. Copy the JDBC driver’s JAR file to the directory created in step 1.

  3. Make the following two changes to the file: <Framework-root>/global_config/overrides/PersistenceService/Transformer/etc/java.conf

    1. Add the following line:

      jvm-global-classpath "${ccd}/../lib/java/**"
    2. Uncomment the following line:

      jvm-location ${JVM_LOCATION}

Configure the Persistence Service

The configuration settings for the persistence data store are held in the configuration file <Framework-root>/global_config/overrides/PersistenceService/Transformer/etc/persistence.conf

File-based persistence is configured by default. If you are using file-based persistence in your development environment, you don’t need to make any configuration changes.

To enable database persistence, make the following changes to persistence.conf:

  1. Comment out enable-file-database

    To comment-out a line, prefix it with '#'.
  2. Uncomment persistence-classid

  3. Change add-database-params/init-string:

    1. Comment out the line beginning: init-string ${ccd}/persistence

    2. Uncomment the line beginning: init-string jdbc:mysql://localhost/rtas

    3. Change the JDBC URL in step (b) above to the JDBC URL for your persistence database

  4. Uncomment driver-name and change the value to the fully-qualified name of the main class of the JDBC driver

  5. Uncomment username and set it to the username of the database account

  6. Uncomment password and set it to the password of the database account

  7. Uncomment table-name and set it to the name of the table to use in the persistence database

  8. Uncomment columns and set it to the names of the two columns in the persistence database table. Specify the name of the key column first, followed by the name of the value column. Separate the column names with a space.

  9. Do not uncomment extra-params

The following table compares configuration settings for file-system and database persistence.

Configuration item File Store Database Store

enable-file-database

TRUE

Comment out

persistence-classid

Comment out

com/caplin/transformer/persistence/JdbcPersistorImpl

add-database-params/init-string

${ccd}/persistence

The JDBC URL for the persistence database.

Example: init-string jdbc:mysql://localhost/rtas

add-database-params/driver-name

Comment out

The fully-qualified classname of the main class in the JDBC driver

add-database-params/username

Comment out

The username of the database account

add-database-params/password

Comment out

The password of the database account

add-database-params/table-name

Comment out

The name of the table used to store persisted data.

Example: table-name persistence

add-database-params/columns

Comment out

The names of the 'key' and 'value' columns, in that order and separated with a space.

Example: columns pers_key pers_val

add-database-params/extra-params

Comment out

DEBUG

Here’s the default persistence.conf configuration file edited to enable database persistence using a local MySQL database called 'rtas':

#
# Override configuration for Persistence Service module.
#
# $Id: //CDev/V-6_2/rtas/blades/PersistenceService/overrides/PersistenceService/Transformer/etc/persistence.conf#3 $
#

log-level       INFO

# enable-file-database

persistence-classid com/caplin/transformer/persistence/JdbcPersistorImpl

add-database-params
#  init-string    ${ccd}/persistence
   init-string    jdbc:mysql://localhost/rtas
   driver-name    com.mysql.jdbc.Driver
   username       root
   password       password
   table-name     persistence
   columns        pers_key pers_val
   extra-params   DEBUG
end-database-params

Additional notes for clustered deployments

To use the Persistence Service in a clustered deployment, activate and configure the Persistence Service on each node in the cluster.

By default, any change to a persistent object results in a write to the shared database, including changes caused by background replication between nodes in a cluster. When a persistent object is changed in a clustered deployment of Transformer, the change is written to the shared database by every node in the cluster. The repeated writes are unnecessary. To stop each cluster node from persisting changes caused by cluster replication, add the following configuration option to <Framework-root>/global_config/overrides/PersistenceService/Transformer/etc/persistence.conf on each node in the cluster.

Configuring permissioning

To allow users to interact with the Persistence Service Client, configure Liberator’s Permissioning Service according to the specification below. For more information about permissioning, see User Authentication and Permissioning.

Rules

Define rules that trigger when users contribute (publish) to the Persistence Service Client’s CONTROL and RECORD subjects. The strings used for the permission namespace and for the action are arbitrary; you are free to customise them to your own naming scheme.

Message Subject Rule Type Permission Namespace Action Action Ref Product Ref

^/PRIVATE/PERSISTENCE/%u/CONTROL

WRITE

PERSISTENCE_SERVICE

PERSISTENCE_CONTROL

-

ALL_PRODUCTS

^/PRIVATE/PERSISTENCE/%u/RECORD/.*

WRITE

PERSISTENCE_SERVICE

PERSISTENCE_RECORD

-

ALL_PRODUCTS

Permissions

Assign users the following permissions.

Product Set Permission Namespace Action Authorisation

.*

PERSISTENCE_SERVICE

PERSISTENCE_CONTROL

ALLOW

.*

PERSISTENCE_SERVICE

PERSISTENCE_RECORD

ALLOW

^/PRIVATE/PERSISTENCE/%u/CONTROL

-

VIEW

ALLOW

^/PRIVATE/PERSISTENCE/%u/RECORD/.*

-

VIEW

ALLOW


See also: