Creating a RET Adapter

The RET Adapter Suite contains a set of integration adapters that are specifically designed to integrate with a RET system and supply RET data to the Caplin Platform. There are six points where integration occurs:

  1. Calendar Adapter
  2. LBN Adapter
  3. Orders Adapter
  4. Permissions Adapter
  5. Pricing Adapter
  6. Trading Adapter

The role of each adapter is to respond to outgoing requests from the Caplin Platform on RET related data or actions, and to act accordingly by communicating with the RET system and providing a response. This communication between an adapter and a RET system is managed by an integration layer that sits between an adapter and a RET system - view this diagram for an illustration. The integration layer provides ​easy access and connectivity to Reuters Trading API (TrAPI), as well as other RET specific APIs that are used to integrate external systems. The Caplin integration layer converts TrAPI layer messages to Caplin messages (and vice versa) and also handles the connection management with RET systems.  

Objectives

In this training module we will take an in depth look at the Calendar Adapter. We will use this adapter as an example for configuring a connection to live data streamed via RET. You will need to configure the TrAPI connectivity for the Calendar Adapter in order to establish a connection to a RET system.

You are going to learn to set up an operational Calendar Adapter, where you will:

  • Configure and connect the adapter to TrAPI (Reuters Trading API)
  • Integrate the adapter with the Caplin Platform, and deploy it as a blade into the Deployment Framework
  • Test that the adapter is working as expected
  • Take what you have learned in the Calendar Adapter example and repeat the steps for all other adapters

Prerequisites

You will need:

  1. Caplin Integration Suite Libraries
  2. Caplin's front-end web trading application (FX Professional Motif) installed with the following parameters set: liberatorHost = available
  3. The RET Adapter Toolkit zip file
  4. RET Libraries: trapi-lib, trapi-rates-plugin, lbn, admin
  5. RET Licences: trapi.lic and lbn.lic
  6. Before starting this training module, a RET connection with the set up recommendations outlined in the table below should be implemented. These recommendations should be addressed by someone that is experienced in using the RET Admin applet.

    Recommended RET connection setup
    Adapter  Username  User Type Modifier1 Required?
    Calendar Adapter

    {env}_cal_client
    {env}_cal_admin

    Proxy Client
    Administrator  

     
    LBN {env}_lbn_proxy   
    {env}_lbn_admin  

    Bank Node
    Administrator 

     
    Pricing  {env}_rates_proxy 
    {env}_rates_admin 

    Proxy Client Administrator  

     
    Trading 

    {env}_fx_proxy_1 
    {env}_fx_proxy_2
    {env}_fx_admin 

    Proxy Client
    Proxy Client
    Administrator 

     
    Orders  {env}_monitor_proxy
    {env}_order_proxy_1
    {env}_order_proxy_2
    {env}_order_admin

    Proxy Client
    Proxy Client 
    Proxy Client
    Administrator

     
    Permissioning  {env}_perms_proxy 
    {env}_perms_admin 

    Proxy Client Administrator  

     
    Clients  {env}_client_1 
    {env}_client_2  
    {env}_client_3  
    {env}_client_4  
    {env}_client_5  
    {env}_client_6  
    {env}_client_7  
    {env}_client_8  
    {env}_client_9  
    {env}_client_10  

    Client

    Yes


     

     

    SalesTrader Clients  {env}_client_st Client No

    1 - Modifiers: This is a config option that is set in the RET Admin applet. A Modifier adjusts rates incoming from RET that are presented to the client. The rate is adjusted to take into account various costs and risks.

Caplin Integration Suite (CIS) Libraries

Ensure you have a copy of the current CIS libraries.

Creating a Workspace

You will need to set up a development environment by preparing a workspace and tools. The first step is to create an adapter blade — this is an adapter that is packaged within a blade —  that includes all the necessary RET configuration. As a starting point, we can simply copy a ready-made example that is supplied with the RET Adapter Toolkit:

Follow the steps below:

  1. Open up your IDE and create a new Java Project with the name CalendarAdapter. It is important that you use this particular name as it is referenced in the start.jar file. Press Finish when you are done.
  2. Now copy over the contents of <RETAdapterToolkit>/examples/CalendarAdapter/ into your newly created project.

  3. Next, navigate to the directory <RETAdapterToolkit>/lib/ and copy all individual files in here over to the existing lib/ directory of the CalendarAdapter workspace <workspace>/CalendarAdapter/Blade/DataSource/lib/. Also copy over all individual files contained inside the third-party folder. 

    Note: All files in <workspace>/CalendarAdapter/Blade/DataSource/lib/ must be at the same directory level, meaning no sub-directories such as folders should exist in there.
  4. You will need to copy in other Caplin library dependencies that were provided separately, these are: FXIntegrationAPI, datasource-java, permissioning-common, permissioning-datasource and trading-datasource.
  5. You will require three .jar files — trapi-lib, trapi-rates-plugin, admin —  these should also be copied into <workspace>/CalendarAdapter/Blade/DataSource/lib/.
  6. Copy the RET licence - trapi.lic - into your CalendarAdapter overrides folder in <workspace>/CalendarAdapter/Blade/overrides/CalendarAdapter/etc.
  7. Change your build path by right-clicking on your project and selecting -> Properties -> Java Build Path
  8. Under the first tab of the Properties dialog box that is labelled Source, change the source folder from src to src/main/java. Do this by clicking on the Add Folder button, expanding the src folder down to the Java folder and then enabling the Java folder by selecting the checkbox. Finally, remove the old src folder and Press OK when you are done. Note: We use a maven based file structure.

  9. Next, you are going to add the libraries that are required to compile your project using the build path set up previously. Look under the Libraries tab in your Java Build Path and press the Add Jars button. Then navigate to the directory RetAdapterToolkit/examples/<Adapter>/Blade/Datasource/lib and add all the files in this folder by selecting all. 

  10. Finally, you can begin compiling your project using two methods:
    1. In you IDE select your build project command (Eclipse: Menu -> Project -> Build/ Intellj: Menu -> Build -> Build Project)
    2. You can also use the projects gradle tasks to compile your project. Do this by opening a command-line interface and using the projects gradle build command or run it in your IDE:

      $ ./gradlew clean build

The directory <RETAdapterToolkit>/lib/ will contain dependencies that are required when compiling an adapter blade. These dependencies consist of RETAdapterToolkit specific libraries as well as third-party libraries. Besides these, there are other dependencies that an adapter will require. You would need Caplin platform libraries, namely FXIntegrationAPIdatasource-javapermissioning-commonpermissioning-datasource and trading-datasource.

Furthermore you need RET specific libraries,  namely trapi-libtrapi-rates-plugin, admin - these are TrAPI libraries that are supplied by Reuters, and required when connecting to a Reuters Electronic Trading (RET) System. You can satisfy dependency requirements for an adapter using the Caplin Integration Suite blade tool - where dependencies are kept in the lib/ directory of an adapter <workspace>.

There is one versions of the <RETAdapterToolkit> that exist in this folder, which should be compatible with your version of Reuters Trading API (TrAPI) that the adapters intend to connect to. A list of all library dependencies can be found in the RETAdapterToolkit.pom file.

Once your adapter has been built, you will then configure it to connect to a live RET system via TrAPI - see the next section. 

Connecting to Reuters Trading API (TrAPI)

The Calendar Adapter must be configured to connect to Reuters Trading API (TrAPI) in order to communicate with a RET system. You are going to compile the code contained within the adapter blade using the blade tool that is available with the Caplin Integration Suite. You will then enable live RET data capture for all the Adapters by configuring the TrAPI connection properties.

Note: usernames and passwords used in the following connection configs are examples only, and each user may only connect once.

For the calendar adapter, you can enable live Calendar data by replacing the contents of Blade/overrides/CalendarAdapter/etc/trapi-connection.properties with the following values.

Note: Host and Port must refer to the host and port of your SCS Relay, and any usernames must match the username/passwords provided to you.
[common]
host=<SCSRelay Host>
port=<SCSRelay Port>

[CalendarConnection]
username={env}_cal_proxy
password=<password>

[AdminConnection]
username=<{env}_cal_admin
password=<password>
order_type=admin

Connecting to the Caplin Platform

After we have configured the TrAPI connectivity and set up the RET connection, we will need to configure the Adapter so that it can communicate RET data to Liberator or Transformer (Caplin's back-end systems).

Open the file <workspace>/CalendarAdapter/global_config/hosts-defns.conf and set the name of each hostname to the location of your Caplin Platform deployment. The hostnames by default should be localhost unless your Platform deployment is on a remote machine, then use the appropriate host name.

hosts-defns.conf is used to generate the file hosts.conf, both files reside in the same location. If you have used the Project templates to generate a new project follow the instruction under: Running your adapter within an IDE. This will not be a problem in deployments that use the Deployment Framework.

Exporting the Platform Configuration

Nearly there! As we have previously configured the RET adapters, you will need to export all configuration and deploy the adapters into the Caplin Platform. The Platform is managed by the Deployment Framework. The Deployment Framework enables you to deploy and run these various components via an easy-to-use command line interface.

To deploy your adapter's configuration to your Deployment Framework (DFW), follow the steps below:

  1. From the root directory of your DFW, run the command below to stop all running Platform components:

    ./dfw stop
  2. Start by copying the file <RETAdapterToolkit>/examples/config into <DeploymentFramework>/global_config.

    Note: This will change the Deployment Framework's default configuration to the configuration required by the RET Adapter Toolkit.
  3. Now export the new configuration:

    Execute the following gradle command either from your IDE or from your console:

    ./gradlew assemble CalendarAdapter --config-only

    The new configuration blade is created under your project's build/distributions directory.

    Ensure that your overrides configuration was deployed correctly, otherwise you will have to manually copy your Blade/overrides/CalendarAdapter over into your DeploymentFramework/global_config/overrides/CalendarAdapter.

  4. Copy the blade ~/src/<Adapter>/build/distributions/<Adapter>-version.zip to your DFW's kits directory:

    cp ~/src/build/distribution/<Adapter>-version.zip ~/DeploymentFramework-version/kits

  5. Finally, you will deploy the configuration into your Deployment Framework. Do this by opening a Cygwin console on your machine and navigating to your Deployment Framework's topmost directory.​ Then execute the following command:

     ./dfw deploy

    This will have configured the Deployment Framework to send any requests such as calendar data request directly to the Calendar Adapter.

    Now start your Deployment Framework using Cygwin. Do this by navigating to the topmost directory of the Deployment Framework and executing the following command:

    ./dfw start

Testing the Adapter

If you want to run your Calendar Adapter within IDE, ensure that you get the run configuration correct.

Follow the steps below:

  1. Select the Arguments tab in the Run Configurations dialog box and look under Wokring directory. Next to Other: input the correct working directory-> ${workspace_loc:CalendarAdapter/Blade/DataSource}
  2. Under the Environments tab, add the following variables:
    • CONFIG_BASE = ../../global_config/ (points to the right configuration directory)
    • LOG_USE_PARENT_HANDLERS = true (to display the logs in the console)

Now that we have our Calendar Adapter deployed into the Caplin Platform (the back end) - let's now test the adapter by running it in your IDE

  1. Using the blade tool's run command that is available with the Caplin Integration Suite, we will run the adapter. Open a command-line tool and execute the following:

    ​java -jar <cis-blade-toolkit> run CalendarAdapter -m com.caplin.example.ret.CalendarAdapterExample
Note: You can use the above step to confirm that the adapter is working. Finally, use <CTRL+ C> to shut-down the adapter.

Troubleshooting the Adapter

If at any point you have trouble testing the adapter, the debugging steps outlined below can be helpful.

Checklist:

  1. Use the Liberator status page on http://localhost:18080/status to determine whether the adapter is UP or DOWN.
  2. Check the console output; unrecoverable exceptions will be displayed here.
  3. Check the adapter logs in <workspace>/CalendarAdapter/Blade/DataSource/var/, in particular the all-calendar-adapter.log (you can find an all-<adapter_name>.log). Any working adapter should log the message Set status to <UP>. If the adapter status is <DOWN> or no message has been logged, this means the adapter is not ready to receive requests. This could be the result of an error, or the adapter may still be initialising. An analysis of the logs will reveal the cause of the problem. Some common problems include:
    • A failed connection to Reuters Trading API (TrAPI)
    • The network is unavailable
    • The TrAPI licence is invalid - these issues can be identified through any exceptions that have been logged in the log files.​​
  4. Test to see if the Calender Adapter is sending data by opening the Liberator Explorer. Navigate to http://localhost:18080/ and click on Diagnostics above, then click on the liberatorexplorer URL link. Now, in the field labelled 'Record' type /CALENDAR/TENORDATES/GBPUSD and press Go.

    You should now see some columns and a single row. View the 'Tenor' column to see how the values are displayed. In the next training module we will look at ways of modifying these values.

Deploy the other Adapters

To complete training, you will need to deploy and configure the other Adapters so that we have the entire Adapter Suite running. 

Follow the steps below:

  1. Unzip the provided RETAdapterSuite.zip
  2. Copy all files from RETAdapterSuite/blades, except for the Calendar Adapter, into your DeploymentFramework/kits folder
  3. Now open up Cygwin and navigate to your DeploymentFramework's topmost directory. Then deploy the adapters using the command./dfw deploy
  4. Before we can start the Deployment Framework we need to configure the RET connection for each adapter. Follow the next steps.

Deploy RET Licences

For each of the adapters, copy the trapi.lic into the overrides folder. Additionally, for the LBNAdapter you will also need to copy lbn.lic as well.

Configure the connection properties for Reuters Trading API 

You will need to configure your adapters within your Deployment Framework, this is usually done in the overrides folder of each adapter. You can find this folder under DeploymentFramework/global_config/overrides.

Follow the steps below:

  1. To enable live data capture for the Execution and Historic Blotter, update the contents of the file global_config/overrides/LBNAdapter/etc/trapi-connection.properties with the following values:

    [common]
    host=<SCSRelay Host>
    port=<SCSRelay Port>
    
    [LBNConnection1]
    username={env}_lbn_proxy
    password=
    order_type=LBN
    
    [AdminConnection]
    username={env}_lbn_admin
    password=
    order_type=admin
    
  2. To enable live pricing data, update the contents of the file global_config/overrides/PricingAdapter/etc/trapi-connection.properties with the following values:

    [common]
    host=<SCSRelay Host>
    port=<SCSRelay Port>
    
    [RatesConnection]
    username={env}_rates_proxy
    password=
    
    [AdminConnection]
    username={env}_rates_admin
    password=
    order_type=admin
    
  3. To enable live trading, update the contents of the file global_config/overrides/FxTradingAdapter/etc/trapi-connection.properties with the following values:

    [common]
    host=<SCSRelay Host>
    port=<SCSRelay Port>
    
    [TradingConnection1]
    username={env}_fx_proxy_1
    password=
    use_for_trading=true
    
    [AdminConnection]
    username={env}_fx_admin
    password=
    order_type=admin
    
  4. To enable the retrieval of live permissions from RET, update the contents of the file global_config/overrides/PermissioningAdapter/etc/trapi-connection.properties with the following values:

    [common]
    host=<SCSRelay Host>
    port=<SCSRelay Port>
    
    [AdminConnection]
    username={env}_perms_admin
    password=
    order_type=admin
    
  5. To enable live Orders trading, update the file global_config/overrides/LimitOrderAdapter/etc/trapi-connection.properties with the following values:

    [common]
    host=<SCSRelay Host>
    port=<SCSRelay Port>
    
    [TradingConnection1]
    username={env}_order_proxy_1
    password=
    use_as_monitor=false
    order_type=limit
    
    [MonitorConnection]
    username={env}_monitor_proxy
    password=
    use_as_monitor=true
    order_type=limit
    
    [AdminConnection]
    username={env}_order_admin
    password=
    order_type=admin
    

You should now be able to start all the adapters using the command ./dfw start

Troubleshooting the Adapters

If at any point you have trouble testing an adapter, the debugging steps outlined below can be helpful.

Checklist:

  1. Use the Liberator status page on http://localhost:18080/status to determine whether the adapter is UP or DOWN.
  2. Check the console output; unrecoverable exceptions will be displayed here.
  3. Check the adapter logs in <workspace>/Adapter/Blade/DataSource/var/, in particular the all-*-adapter.log (you can find an all-<adapter_name>.log). Any working adapter should log the message Set status to <UP>. If the adapter status is <DOWN> or no message has been logged, this means the adapter is not ready to receive requests. This could be the result of an error, or the adapter may still be initialising. An analysis of the logs will reveal the cause of the problem. Some common problems include:
    • A failed connection to Reuters Trading API (TrAPI)
    • The network is unavailable
    • The TrAPI licence is invalid - these issues can be identified through any exceptions that have been logged in the log files.​​

Review

The techniques covered in this training module can be used in the development of all RET Adapters.