Installing the Refiner Service

In Transformer, data transformation services are implemented in Transformer Modules. Caplin provides a Transformer module called Refiner that allows large lists of data (containers) to be efficiently filtered and sorted in real time on behalf of client applications. Refiner is supplied as a Service blade that's deployed in the Caplin Platform Deployment Framework. This tutorial shows you how to deploy the Refiner blade and use it to sort and filter container responses.

Once you've completed the steps on this page, you should be familiar with how to deploy Refiner and how to request filtered and sorted data via Transformer.

Tip: In these instructions you'll be using the dfw command of the Deployment Framework. Before entering any dfw command as ./dfw <command-name>, make sure your current (working) directory is set to the Deployment Framework's topmost directory.
In file paths, we've called this topmost directory <Framework-root> 

For a list of dfw commands, click here.

Contents:

Requirements

See the table below for a list of products and components required by Refiner 6.2 and Refiner 7.

  Java Deployment Framework Transformer
Refiner 6.2 7+ 6.2 6.2
Refiner 7.0 8 7 7

The Refiner Service is a licenced Caplin product. To confirm that your Transformer is licensed to use the Refiner Service, look for the following line in your Transformer licence file, <Framework_root>/global_config/licenses/license-transformer.conf.

module jtm transformer

To acquire a licence to run the Refiner Service, contact Caplin Support.

Installation instructions

To install Refiner, follow the instructions below:

  1. Install a Deployment Framework, Liberator, and Transformer
  2. Deploy the Refiner Service blade

1. Install a Deployment Framework, Liberator and Transformer

Make sure you've installed a Deployment Framework and have deployed a Liberator and Transformer (see How can I... Deploy Platform components to the Framework).

The Framework has a built-in demonstration Adapter blade called DemoDataSource that you'll use to show that Refiner is working. Make sure the DemoDataSource blade is active, by running the following dfw command from the <Framework-root> directory:

./dfw activate DemoDataSource

You can verify that the blade has been activated, by running the command:

./dfw versions

2. Deploy the Refiner Service blade

The Refiner Service blade is supplied in a zip file with a name like CPB_RefinerService-<version>-<build_number>.zip.

To deploy the Refiner Service blade, follow the instructions in To deploy the blade kits and Check what's been deployed in How can I... Deploy Platform components to the Framework.

Check the Refiner service is available

Follow the instructions below to check the Refiner Service is available:

  1. Start the Liberator, Transformer and DemoDataSource adapter, by running ./dfw start from the <Framework-root> directory.
  2. Log on to Liberator from a web browser and navigate to the status page (for details of how to do this, see Check the Liberator is running correctly in Installing Liberator).

    The Liberator should recognise the following data services and DataSources. In particular, the status page should show a data service for the Refiner Service, as highlighted in red here:

Refiner service status shown on Liberator status page

  1. Stop the Liberator, Transformer and DemoDataSource adapter, by running ./dfw stop from the <Framework-root> directory. This is so you can examine various log files to check Transformer's working correctly.
     
  2. Examine the log file <Framework-root>/servers/Transformer/var/jtm.log

If the Transformer is working correctly, this log contains the version number of the Transformer Module library:

<YYYY/MM/DD-HH:MM:SS.NNN> +0000: INFO: Java Transformer module (6.2.0-297741) starting up
<YYYY/MM/DD-HH:MM:SS.NNN> +0000: INFO: Loading user class with identifier jtm
<YYYY/MM/DD-HH:MM:SS.NNN> +0000: CRIT: Thread [main/1]: Product: Java Transformer Module
Version        : 6.2.3-bbbbbb
Build Date     : dd-Mon-yyyy 
Build Time     : hh:mm
Build Number   : bbbbbb
Copyright      : Copyright 1995-2014 Caplin Systems Ltd

In the same log file, you should also see Caplin Refiner registering as a provider for the namespace it uses:

2011/12/16-15:41:05.703 +0000: INFO: Registering as a provider of </FILTER/*>
  1. Examine the log file <Framework-root>/servers/Transformer/var/refiner.log
    If Caplin Refiner is working correctly, this log contains its version number:
<YYYY/MM/DD-HH:MM:SS.NNN> +0000 - Thread [main/1]: SEVERE: Refiner 6.2.3-bbbbbb
  1. If everything looks ok, restart the Liberator, Transformer and DemoDataSource adapter, by running: ./dfw start
     
  2. Log on to Liberator from a web browser and navigate to the Liberator Explorer page:
    1. From the menu on the Liberator's home page, select Diagnostics
    2. On the Diagnostics page that's displayed, select the Liberator Explorer link.
Tip: For more about using Liberator Explorer, see How can I... Use Liberator Explorer to request and send data.

Requesting an unfiltered container

The DemoDataSource adapter can supply a container called /EXAMPLES/PRICING/CONTAINERS/EQUITIES that's populated with stock prices.

Use Liberator Explorer to request /EXAMPLES/PRICING/CONTAINERS/EQUITIES. This returns the un-filtered and un-sorted container:

Unfiltered container of stock prices

Requesting a filtered container

In Liberator Explorer, request /FILTER/EXAMPLES/PRICING/CONTAINERS/EQUITIES?filter=(BestBid>20)

Liberator routes this request to Transformer, because its RefinerService1 data service contains the include-pattern ^/FILTER/ (have a look in the Liberator configuration file for the Refiner Service blade at <Framework-root>/kits/CPB_RefinerService/CPB_RefinerService-<version>/Liberator/etc/rttp.conf).

The /FILTER prefix directs Transformer to pass the request on to its Refiner module, which then returns just those instruments whose BestBid field is greater than 20

Stock prices filtered by Refiner

Requesting a sorted container

In Liberator Explorer, request /FILTER/EXAMPLES/PRICING/CONTAINERS/EQUITIES?sort=FullName:text:ascending

Now Refiner sorts the instruments in ascending alphabetical order of the FullName field:

Stocks sorted by name by Refiner
 

Warning! Avoid filtering or sorting on record fields that are subject to frequent updates. Frequent updates can have an adverse affect on the performance of Caplin Refiner, Liberator, and the requesting clients.

See also: