Activating the Persistence Service

The Persistence Service provides Transformer modules and StreamLink clients with a key-value store, persistent across server restarts and shared between nodes in a Transformer cluster.

The Persistence Service comprises two built-in blades that are included with Transformer from version 6.2.3. Neither blade is active by default.

The Persistence Service is required by the following Transformer services: Watchlist, Charting, and High-Low.

Contents:

Activating the blades
Configuring the Persistence Service
Configuring permissioning

Activating the blades

There are two Persistence Service modules:

PersistenceService: This module provides persistence services to Transformer modules, including the PersistenceServerClient module. See the Java package com.caplin.transformer.persistence.
PersistenceServiceClient: This module provides persistence services to StreamLink clients. It requires the PersistenceService module to be active.

To activate the Persistence Service blades, run the commands below from the root of your Deployment Framework:

$ ./dfw activate PersistenceService
$ ./dfw activate PersistenceServiceClient

Configuring the Persistence Service

Follow the instructions in this section to configure the Persistence Service.

Deciding which datastore to use

Transformer’s Persistence Service supports both local and shared datastores.

Local datastore (filesystem)

Data is persisted to Transformer’s local filesystem. This is the default configuration and is suitable for development use only.

File-backed persistence is unsupported in production environments, and, from version 7 of the Persistence Service, file-backed persistence cannot be used in Transformer clusters.

Shared datastore (database)

Data is peristed to a SQL database server. This configuration is supported for use in production use.

This datastore can be shared between instances of Transformer and so is suitable for clustered deployments.

To configure database-backed persistence:

Create the database table to store persistence data.
Add the database’s JDBC driver to Transformer’s classpath.
Configure the Persistence Service, providing connection details to the database.

Configuring a shared datastore

Create the database table

Choose a SQL database that supports JDBC access over the network using text-based authentication credentials.

Create a table with two text columns, one to store persistence keys and the other to store persistence values. Define the column that stores the persistence keys as the primary key of the table. You may choose any name for the table and its columns: the table name and column names are configurable in the Persistence Service’s configuration file (see Defining the persistence configuration items).

Here’s an example MySQL table definition that uses the table and column names defined in the Persistence Service’s default configuration file:

CREATE TABLE persistence (
    pers_key VARCHAR(100),
    pers_val TEXT,
    PRIMARY KEY (pers_key)
);

You may need to increase the width of the 'pers_key' column if a service or client uses keys greater than 100 characters in length.

Add the database’s JDBC driver to Transformer’s classpath

To add the database’s JDBC driver to Transformer’s classpath:

Create the following directory: <Framework-root>/global_config/overrides/PersistenceService/Transformer/lib/java
Copy the JDBC driver’s JAR file to the directory created in step 1.
Make the following two changes to the file: <Framework-root>/global_config/overrides/PersistenceService/Transformer/etc/java.conf
1. Add the following line:
  jvm-global-classpath "${ccd}/../lib/java/<jar-filename>"
2. Uncomment the following line:
  jvm-location ${JVM_LOCATION}

Configure the Persistence Service

The configuration settings for the persistence data store are held in the configuration file <Framework-root>/global_config/overrides/PersistenceService/Transformer/etc/persistence.conf

File-based persistence is configured by default. If you are using file-based persistence in your development environment, you don’t need to make any configuration changes.

To enable database persistence, make the following changes to persistence.conf:

Comment out enable-file-database

To comment-out a line, prefix it with '#'.
Uncomment persistence-classid
Change add-database-params/init-string:
1. Comment out the line beginning: init-string ${ccd}/persistence
2. Uncomment the line beginning: init-string jdbc:mysql://localhost/rtas
3. Change the JDBC URL in step (b) above to the JDBC URL for your persistence database
Uncomment driver-name and change the value to the fully-qualified name of the main class of the JDBC driver
Uncomment username and set it to the username of the database account
Uncomment password and set it to the password of the database account
Uncomment table-name and set it to the name of the table to use in the persistence database
Uncomment columns and set it to the names of the two columns in the persistence database table. Specify the name of the key column first, followed by the name of the value column. Separate the column names with a space.
Do not uncomment extra-params

The following table compares configuration settings for file-system and database persistence.

Configuration item File Store Database Store

Configuration item	File Store	Database Store
`enable-file-database`	`TRUE`	Comment out
`persistence-classid`	Comment out	`com/caplin/transformer/persistence/JdbcPersistorImpl`
`add-database-params/init-string`	`${ccd}/persistence`	The JDBC URL for the persistence database. Example: `init-string jdbc:mysql://localhost/rtas`
`add-database-params/driver-name`	Comment out	The fully-qualified classname of the main class in the JDBC driver
`add-database-params/username`	Comment out	The username of the database account
`add-database-params/password`	Comment out	The password of the database account
`add-database-params/table-name`	Comment out	The name of the table used to store persisted data. Example: `table-name persistence`
`add-database-params/columns`	Comment out	The names of the 'key' and 'value' columns, in that order and separated with a space. Example: `columns pers_key pers_val`
`add-database-params/extra-params`	Comment out	`DEBUG`

enable-file-database

TRUE

Comment out

persistence-classid

Comment out

com/caplin/transformer/persistence/JdbcPersistorImpl

add-database-params/init-string

${ccd}/persistence

The JDBC URL for the persistence database.

Example: init-string jdbc:mysql://localhost/rtas

add-database-params/driver-name

Comment out

The fully-qualified classname of the main class in the JDBC driver

add-database-params/username

Comment out

The username of the database account

add-database-params/password

Comment out

The password of the database account

add-database-params/table-name

Comment out

The name of the table used to store persisted data.

Example: table-name persistence

add-database-params/columns

Comment out

The names of the 'key' and 'value' columns, in that order and separated with a space.

Example: columns pers_key pers_val

add-database-params/extra-params

Comment out

DEBUG

Here’s the default persistence.conf configuration file edited to enable database persistence using a local MySQL database called 'rtas':

#
# Override configuration for Persistence Service module.
#
# $Id: //CDev/V-6_2/rtas/blades/PersistenceService/overrides/PersistenceService/Transformer/etc/persistence.conf#3 $
#

log-level       INFO

# enable-file-database

persistence-classid com/caplin/transformer/persistence/JdbcPersistorImpl

add-database-params
#  init-string    ${ccd}/persistence
   init-string    jdbc:mysql://localhost/rtas
   driver-name    com.mysql.jdbc.Driver
   username       root
   password       password
   table-name     persistence
   columns        pers_key pers_val
   extra-params   DEBUG
end-database-params

Additional notes for clustered deployments

To use the Persistence Service in a clustered deployment, activate and configure the Persistence Service on each node in the cluster.

By default, any change to a persistent object results in a write to the shared database, including changes caused by background replication between nodes in a cluster. When a persistent object is changed in a clustered deployment of Transformer, the change is written to the shared database by every node in the cluster. The repeated writes are unnecessary. To stop each cluster node from persisting changes caused by cluster replication, add the following configuration option to <Framework-root>/global_config/overrides/PersistenceService/Transformer/etc/persistence.conf on each node in the cluster.

dont-persist-cluster-messages   TRUE

Configuring permissioning

To allow users to interact with the Persistence Service Client, configure Liberator’s Permissioning Service according to the specification below. For more information about permissioning, see User Authentication and Permissioning.

Rules

Define rules that trigger when users contribute (publish) to the Persistence Service Client’s CONTROL and RECORD subjects. The strings used for the permission namespace and for the action are arbitrary; you are free to customise them to your own naming scheme.

Message Subject	Rule Type	Permission Namespace	Action	Action Ref	Product Ref
^/PRIVATE/PERSISTENCE/%u/CONTROL	WRITE	PERSISTENCE_SERVICE	PERSISTENCE_CONTROL	-	ALL_PRODUCTS
^/PRIVATE/PERSISTENCE/%u/RECORD/.*	WRITE	PERSISTENCE_SERVICE	PERSISTENCE_RECORD	-	ALL_PRODUCTS

Message Subject

Rule Type

Permission Namespace

Action

Action Ref

Product Ref

^/PRIVATE/PERSISTENCE/%u/CONTROL

WRITE

PERSISTENCE_SERVICE

PERSISTENCE_CONTROL

ALL_PRODUCTS

^/PRIVATE/PERSISTENCE/%u/RECORD/.*

WRITE

PERSISTENCE_SERVICE

PERSISTENCE_RECORD

ALL_PRODUCTS

Permissions

Assign users the following permissions.

Product Set	Permission Namespace	Action	Authorisation
.*	PERSISTENCE_SERVICE	PERSISTENCE_CONTROL	ALLOW
.*	PERSISTENCE_SERVICE	PERSISTENCE_RECORD	ALLOW
^/PRIVATE/PERSISTENCE/%u/CONTROL	-	VIEW	ALLOW
^/PRIVATE/PERSISTENCE/%u/RECORD/.*	-	VIEW	ALLOW

Product Set

Permission Namespace

Action

Authorisation

PERSISTENCE_SERVICE

PERSISTENCE_CONTROL

ALLOW

PERSISTENCE_SERVICE

PERSISTENCE_RECORD

ALLOW

^/PRIVATE/PERSISTENCE/%u/CONTROL

VIEW

ALLOW

^/PRIVATE/PERSISTENCE/%u/RECORD/.*

VIEW

ALLOW

See also:

Persistence configuration