The Java Transformer Module (JTM) API provides Java developers with the tools necessary to perform business rules and calculations on data updates being passed between applications connected to the DataSource API (DataSource peers such as Caplin Liberator servers). The API for creating DataSource providers and subscriptions is very similar to the Java DataSource API. The JTM supports the same DataSource Message types and shared API structure.

For example, you can use the SDK to do the following:

  • calculate the top 10 shares in terms of price, yield, or number of transactions per hour;
  • log incoming data to a database, and if a corrected or marked trade appears in a feed determine whether that is passed on or not;
  • simplify updates so that if multiple updates are contained within the update packet, only the last (and hence most significant) is sent out;
  • perform predetermined tasks according to market conditions, such as sending initialisation messages when markets open and end-of-day calculations when they close;
  • process suspensions in trading and determine what to do with the data in alignment with market rules.

Each calculation or set of calculations is performed by individual modules connected to a central core. The Transformer Core distributes incoming data to the modules that have requested it. It is also a DataSource peer.

A module for Caplin Transformer takes the form of a dynamic library which is loaded on startup or on receipt of a UDP command. A Java module is loaded into a VM started up by the Transformer Core, configured from a file. When a module requests data, the data sink element of the Transformer Core forwards the request to that peer or set of peers which can provide it. When an update occurs, the Core extracts the returned data from the DataSource API and sends it to the module for processing.

The module can then output the transformed data to the Core for distribution.

There are two main steps to using the Transformer Module SDK:

  1. Edit the configuration file transformer.conf so that the Transformer Core can communciate with its modules and other applications connected to the DataSource API.
  2. Use the Java classes provided by the SDK to create the individual business modules to process the data supplied through the Transformer Core.

Using the Transformer SDK for Java

Follow the links on the table below for information on the following topics:

Steps to create a module
Create the main module object
Request the data needed for calculations
Respond to requests from peers
Manipulate incoming data
Publish the transformed data
Configure the Core so that it can load the module
Using UDP commands

Create the main module object

The TransformerModule is the main interface for the Transformer Module Java SDK. This interface must be implemented by a developer, and the resulting class should be the one referenced in the transformer.conf file.

In addition, a reference to the TransformerAccessor passed into the Transformer module initialise method must be kept in order to access the main interfaces needed to communicate with the Transformer Core.

TOP

Request the data required for calculations

In order for Caplin Transformer to supply updated information to a module, the module has to tell the Core that it has interest in a particular set of symbols. The Core then makes an active request for the symbols from DataSource applications connected to the DataSource API, unless a requested symbol is already within the its memory (i.e. another module has requested it), in which case your module will receive the data without it being rerequesting from the data source.

The TransformerAccessor object allows a module to create active and broadcast subscriptions using the Transformer Core. This can be done using familiar DataSource Java API methods, createActiveSubscription and createBroadcastSubscription. The API differs in one way, where it is necessary to call subscribe() on the subscription object returned by the creation call to initialise the subscription.

If you specify wildcards in the object name pattern (for example /LO/* for all symbols on the London Stock Exchange), the data cannot be requested from any peers: your module must wait for the information to come into the Transformer Core. Only if the exact symbol is known can an active request be made to a DataSource peer (for example /LO/BAY). Wildcards are particularly useful for listening for broadcast data however, which will come into the Transformer Core without being actively requested by any module.

Example:

SubscriptionListener subListener = new LondonSubscriptionListener();
TransformerActiveSub activeSubscription = transformerAccessor.createActiveSubscription("/LO/*", subListener);
activeSubscription.subscribe();
public class LondonSubscriptionListener implements SubscriptionListener {
   public void recordUpdated(Subscription subscription, Peer peer, RecordMessage message) {}
   public void recordType2Updated(Subscription subscription, Peer peer, RecordType2Message message) {}
   public void recordType3Updated(Subscription subscription, Peer peer, RecordType3Message message) {}
   public void genericMessageUpdated(Subscription subscription, Peer peer, GenericMessage message) {}
   public void permissionUpdated(Subscription subscription, Peer peer, PermissionMessage message) {}
   public void newsUpdated(Subscription subscription, Peer peer, NewsMessage message) {}
   public void storyUpdated(Subscription subscription, Peer peer, StoryMessage message) {}
   public void containerUpdated(Subscription subscription, Peer peer, ContainerMessage message) {}
   public void pageUpdated(Subscription subscription, Peer peer, PageMessage message) {}
   public void subjectErrorReceived(Subscription subscription, Peer peer, SubjectErrorEvent subjectErrorEvent) {}
   public void subjectStatusReceived(Subscription subscription, Peer peer, SubjectStatusEvent subjectStatusEvent) {}
}

TOP


Respond to requests from peers

The TransformerAccessor object allows a module to register and deregister itself as a provider of data using the createActivePublisher methods. createActivePublisher is very similar to the DataSourceJava equivalent. However, the TransformerPublisher object returned provides extra methods for publishing DataSource Message objects with additional Transformer flags, PublishFlag.

Creating an active publisher:

DataProvider chartingProvider = new ChartingDataProvider();
transformerAccessor.createActivePublisher("/CHART/*", chartingProvider);

public class ChartingDataProvider implements DataProvider
{
    public DataProviderResponse request(RequestEvent requestEvent)
   {
      // code to create a thread to start contributing data when
      // this method has finished executing...
   }
   public DataProviderResponse discard(DiscardEvent discardEvent)
    {
      // code to stop contribution of data for the specified
      // object...
    }
}

TOP


Using object name patterns

Every exchange uses a different set of symbols to identify the various financial instruments that are traded. Object name patterns enable you to search all available symbols and return a list of those fitting your search criteria.

Patterns may contain asterisk "*" and question mark "?" wildcards. The asterisk will match any number of characters, whilst the question mark will match any one character.

For example:

  • a search string of "/LO/*" will return all symbols traded on the London Stock Exchange and a search string of "/*/MSFT" will return the symbols of Microsoft stocks on all available exchanges (assuming constant symbology).
  • the pattern /LO/* will match /LO/VOD, /LO/CAPLIN and /LO/DIR/OBJ1, but will not match /NA/VOD, /DIR/LO/VOD or /LO2/CAPLIN.
TOP

Manipulate data

The DataCache class provides a module access the data that is cached within the Transformer core. This data might be useful for helping with calculations based on an update that has just been received.

Finding updates to manipulate

The latest values for an object can be retrieved from the Transformer Core using the get(objectName) method. The message type can be accessed by calling getMessageType on the returned message object. If necessary, the Message can be cast to the appropriate DataSource message type. To find out which objects are available in the Core, the getObjectNames(objectNamePattern) method can be used.

Example:

Message vodafoneData = transformerAccessor.getDataCache().get("/LO/VOD");
if (vodafoneData != null)
{
    // code to process the cached data...
}

Finding the time of an update

The method getLastUpdateTime(objectName) returns the last time an update was received for this symbol. This can be used to create modules which output data regarding time periods for use in charts.

Copying updates

Once a Message is sent to the Transformer core, its memory is freed. To send new Message updates to the Transformer core, new Messages must be constructed using the publisher MessageFactory object.

Updating fields

Fields can be updated using the DataSource message API methods. For example, a subclass of RecordMessage provides the following methods:

Example:

public class FieldModificationSubscriptionListener implements SubscriptionListener {
   public void recordUpdated(Subscription subscription, Peer peer, RecordMessage message) {
      String rawFieldValue = message.getField(FOREX_WHOLE);
      if (rawFieldValue != null) {
         try {
            double roundedValue = Math.round(Double.parseDouble(rawFieldValue));
            message.setField(FOREX_WHOLE, String.valueOf(roundedValue));
         } catch (NumberFormatException nfe) {
            logger.log(Level.SEVERE, "Non numeric " + FOREX_WHOLE + " field value: " + rawFieldValue);
         }
      }
   }
    ... More SubscriptionListener Callbacks ...
}

TOP

Publish the transformed data

The Caplin Transformer core can output data in the following ways:

  • publish data to DataSource peers connected to the DataSource API; or
  • send responses to requests for data from Caplin Liberators. When the core receives a request, it checks its list of providing modules for a match (modules tell the core which symbols they handle), and if a match is made calls the module. If the module does not handle that symbol then a request is sent out to the data sources and Transformer filters the results and sends them back to Liberator.
Messages can be published by the Transformer using the TransformerPublisher returned when creating an active publisher. It is also possible to publish messages without creating an active publisher. This can be done using the global publisher attained through getGlobalPublisher.

This may be done if Transformer is already registered as a DataSource provider through another Transformer module.

Example:

String objectName = "/LO/CPLN";
Publisher publisher = transformerAccessor.getGlobalPublisher();
RecordMessage record = publisher.getMessageFactory().createRecordType1Message(objectName);
record.setField("CompanyName", "Caplin Systems Limited");
record.setField("Currency", "GBP");
record.setField("Volume", "0");
publisher.publishToSubscribedPeers(record);

TOP


Configure Transformer Core to load the module

To load a Transformer Java Module, the following configuration settings should be added to the Core's configuration file etc/transformer.conf:

  • add-module
    Tells the Core to load a module. Should contain the name of the .so file (jtm_DS5.so) and can also set a flag to determine whether the Core API can be extended to be used within the module.
  • jvm-location
    Location of the libjvm.so file. Should contain the complete path and include the .so/.dll suffix.
  • jvm-global-classpath
    Location of the global classpath. Default is %r/lib/java. To include several classpaths, separate jvm-global-classpath entries should be added for each one.
  • add-javaclass
    Identifies the module to be loaded.

Note: For more information on these and other parameters, and how the Transformer Core is configured, please refer to the Transformer Core Administrator's Guide.

The example configuration below loads a Transformer Module implemented in a Java class, examples.first.FirstTransformerModule.

add-module       jtm_DS5

# Configuration of Java class to be loaded
add-javaclass
    class-name   examples.first.FirstTransformerModule
    class-id     jtm_DS5
    classpath    %r/lib/java/examples.jar
end-javaclass

# Configuration of the JVM
jvm-location          /usr/local/jdk/jre/lib/sparc/libjava.so
jvm-global-classpath  %r/lib/java/transformermodule_DS5.jar

To add other startup options for the JVM, multiple configuration lines beginning jvm-options may be specified. For example, to enable socket debugging on port 9955 the following configuration options could be added:

jvm-options    -Xdebug
jvm-options    -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9955

To load a second Transformer Java Module, make a copy of jtm_DS5.so (found in the lib folder), and add a second add-javaclass entry. The class-id parameter must be set to the name of the copied file (jtm_DS5_2 in the example below).

For example, if jtm_DS5.so were copied as jtm_DS5_2.so, a second module could be loaded by:

add-module       jtm_DS5_2.so

add-javaclass
    class-name   examples.second.SecondTransformerModule
    class-id     jtm_DS5_2
    classpath    %r/lib/java/examples2.jar
end-javaclass

TOP


Using UDP commands

The UDPAccessor allows a module to send UDP commands and to add or remove its own listeners for UDP commands.

Listeners associated with a certain UDP message will be informed each time that command is executed.

If a particular listener is added multiple times for the same command, then it will be informed of a single execution of that command multiple times. When a listener that has been added multiple times for the same command is deleted, only the first occurrence Listeners will be removed. If one particular listener is used to listen to multiple commands, it can identify which command was executed from the first command argument (which is always the command itself).

TOP

UDP commands that may be useful to a Transformer module include the following:

  • memory_write
    tells the Core to write its memory to file. This will be read back when the Transformer is restarted.
  • symbol_publish arg1 arg2 ... argN -
    tells the Core to publish all of the data for the specified objects. The object arguments can either refer to exact object names, or can be object name patterns, including wildcards.
  • source_up peerId
    a command sent from the Core to inform its modules that the specified peer is now up. If your module needs to know about which peers are up, then it should register a listener for the source_up command.
  • source_down peerId
    a command sent from the Core to inform its modules that the specified peer is now down. If your module needs to know about which peers are down, then it should register a listener for the source_up command.

Other commands can be sent by a Transformer module; the Transformer core will send these to any other modules that are listening for them.

Example:

// tell the Transformer Core to write its memory out to disk
transformerAccessor.getUDPAccessor().send("memory_write");

TOP


Logging

TransformerAccessor provides a getLogger() method which returns a java.util.logging.Logger that can be used to write messages to the Transformer module's log file.

Use Level when setting the log level.

Example:

transformerAccessor.getLogger().log(Level.SEVERE, "This is an error message");

TOP

 

Packages 
Package Description
com.caplin.datasource  
com.caplin.datasource.channel
The Channel package consists of interfaces allowing creation and maintenance of Channels that provide a bi-directional communication mechanism between StreamLink clients and DataSource applications
com.caplin.datasource.fields  
com.caplin.datasource.interfaces  
com.caplin.datasource.messaging
Defines the message types that are used by DataSource for Java.
com.caplin.datasource.messaging.container
This package defines the interfaces for sending Container messages.
com.caplin.datasource.messaging.json
This package defines the interfaces for sending JSON messages.
com.caplin.datasource.messaging.mapping
This package defines the interfaces for sending Mapping messages.
com.caplin.datasource.messaging.news
This package defines the interfaces for sending News messages.
com.caplin.datasource.messaging.page
This package defines the interfaces for sending Page messages.
com.caplin.datasource.messaging.permission
This package defines the interfaces for sending Permission messages.
com.caplin.datasource.messaging.record
This package defines the interfaces for sending Record messages.
com.caplin.datasource.messaging.story
This package defines the interfaces for sending Story messages.
com.caplin.datasource.namespace
Namespaces are used to determine if a subject is of interest to an DataProvider.
com.caplin.datasource.publisher
Defines the methods of providing and publishing data that are used by DataSource for Java.
com.caplin.datasource.subscription
Defines the interfaces and classes that are used by a Java DataSource application when it needs to subscribe to data provided by another DataSource application.
com.caplin.jtm  
com.caplin.jtm.exceptions  
com.caplin.jtm.net.udp  
com.caplin.jtm.persistence  
com.caplin.jtm.pipeline  
com.caplin.jtm.wrappers