Load-balance adapters using source affinity

Source affinity is an alternative load-balancing algorithm that distributes channel subscriptions evenly across a set of adapters while maintaining the integrity of user sessions.

Overview

If an integration adapter is designed to work with two or more of a user’s channels, serving data to one channel as a result of activity on another, then load balancing adds an extra layer of complexity. Both of the user’s channels must be allocated to the same adapter instance in the load-balanced set. The standard load-balancing algorithm cannot guarantee this.

Using source affinity, you can:

  • ensure a user’s trading-blotter channel and trade channel are load balanced to the same instance of a trading adapter

  • ensure a user’s order-blotter channel and order channel are load balanced to the same instance of an order adapter

  • ensure a user’s activity blotter channel that combines trades and orders, is load-balanced to the same adapter instances as the user’s trade and order channels

See also Data services : Source affinity.

System requirements

Source affinity requires the following versions of Liberator and Transformer:

Support for source affinity in Liberator and Transformer
Liberator Transformer

Definition at data-service level

6.2.2+

6.2.2+

Definition at source-group level

6.2.6+

6.2.5+

Configuration reference

Source affinity is enabled using the affinity configuration option of the add-source-group DataSource configuration item.

The affinity configuration option of the add-data-service item is deprecated and is supported for backwards compatibility only. Enabling source affinity at source-group level is more flexible and has fewer constraints than enabling source affinity at data service level.

Don’t include the affinity configuration option at both data service and source-group level within the same data service.

Syntax

The affinity configuration option takes the following syntax:

affinity <affinity_key> <regular_expression>
Parameters
Parameter Description

affinity_key

An arbitrary string identifying a set of user-to-adapter affinities

regular_expression

The regular expression to extract the username from a channel’s mapped subject.

Specifying the affinity key

The affinity key plays a role in the persistence of adapter selections. Persisted adapter selections are reused for future channels owned by the same user and configured with the same affinity key.

When an adapter instance is selected for a user’s channel, a record of the selection is made under a combined key of the username and the channel’s affinity key. Combining the username with the affinity key enables the source affinity algorithm to persist more than one adapter selection per user.

DataSource components (Liberator and Transformer) manage independent source-affinity data stores. These data stores are not shared or synchronised. Under conditions of changing adapter availability, it is possible for multiple DataSource components to select different adapter instances for the same source affinity key.

When an adapter selected for a user’s channel fails, the source affinity algorithm selects an alternative adapter instance. The affinity to the old adapter instance is forgotten and the newly-selected adapter is reused for future channels owned by the user and configured with the same affinity key.

To ensure that source groups in separate data services select the same adapter for a user, assign the source groups the same affinity key.

Source groups that are assigned the same affinity key must be identical in terms of the number and order of their adapters.

Example keys

At source-group level, affinity keys are generally named after the set of adapters defined by the source group. For example, 'order-adapters' or 'trading-adapters'.

At data-service level (now deprecated), affinity keys are generally named after a use case. For example, 'ordering' or 'trading'.

Specifying the regular expression

The 'source' in source affinity describes the quality of the algorithm that distributes subscriptions across adapters based not on a quality of where the subscriptions are going to (the adapters) but on a quality of where they have come from (their source). Channels are the only subscriptions that are distinguishable by source; they are private subscriptions and their mapped subjects are distinguishable by the presence of a username or session name. The regular expression parameter is used by source affinity to extract the username from the mapped subject(s) served by a source group.

Follow the guidance below when composing the regular expression to extract the username:

  • Adhere to POSIX Extended Regular Expression (ERE) syntax. Source affinity uses the GNU regular expression library installed on the host operating system.

  • Do not enclose the regular expression with the customary regular-expression delimiter, ‘/’.

  • Do not escape ‘/’ characters

  • Include only one submatch group, which should capture the username. Multiple groups are not supported and will raise an error.

    Some mapped subjects may use the user’s session name (%U in object mapping syntax) as a differentiator; make sure that you capture the username, regardless of whether the differentiator is the user’s username or session name. Session names add the suffix '-n' to the username, where 'n' is an integer. The example regular expression following this list shows one way of limiting a capture to the username only.

  • Do not chain regular expressions with the '|' operator. Chaining regular expressions would require more than one submatch group, and only one submatch group is supported.

  • For guidance on the format of the channel subjects your expression will parse, see the include-pattern configuration options of the containing data service, and the object-map configuration items for the channel subjects.

Example regular expression

The following regular expression complies with the guidance above, and is a good starting point for writing your own regular expression:

/PRIVATE/([^-]+).*/

Procedure

To enable source affinity for two or more channels so that they are served by the same instance of an adapter in a source group, follow the steps below:

  1. Determine the location of the configuration files to edit.

    • If you have deployed your environment using the Caplin Deployment Framework, the configuration files will be in the adapter blade.

    • If you have deployed your environment manually, the configuration files will be in the installation directories of Liberator and Transformer

  2. Determine which data services to edit. For a channel that is routed via Transformer, the definition to edit will be in Transformer’s configuration file (transformer.conf). For a channel that is routed directly to the integration layer, the definition to edit will be in Liberator’s configuration file (rttpd.conf).

  3. Examine the data services identified in step 2. Ensure that the source groups for the adapter are identically configured in terms of the number and order of adapter instances.

  4. Enable source affinity for the adapter’s source group in each data service.

    1. All source groups for the adapter should use the same affinity key, and the affinity key used should be unique to that adapter.

    2. The regular expression used to extract each channel’s username can be different between source groups, but it must extract the same string: the username.

Examples

The two examples below show how to configure two types of activity blotter.

  • An activity blotter that sources data from one adapter (one source group)

  • An activity blotter that sources data from two adapters (two source groups)

The examples assume that trade and order channels connect directly to the integration layer, wheras blotters connect to the integration layer via Transformer.

For the sake of clarity, the example configurations do not include provision for failover or for filtering the activity blotter.

Activity blotter: one source-group

This example demonstrates how to configure an activity blotter that records the activity of a trading adapter.

Configure the trade channel’s data service on Liberator and the trade blotter’s data service on Transformer with the same affinity key.

Liberator configuration

# Object mapping for the trade channel
object-map   "/PRIVATE/TRADE/FX%1"   "/PRIVATE/%u/TRADE/FX%1"

# Object mapping for the activity blotter
object-map   "/PRIVATE/FX/BLOTTER/ACTIVITY%1"   "/PRIVATE/%u/FX/BLOTTER/ACTIVITY%1"
object-map   "/PRIVATE/FX/BLOTTER/ITEM/%1"   "/PRIVATE/%u/FX/BLOTTER/ITEM/%1"

# Trade Channel
add-data-service
   service-name         fx-trading
   include-pattern      "^/PRIVATE/[^/]*/TRADE/FX"

   add-source-group
      required true
      affinity          fx-trading-adapters "/PRIVATE/([^-]+).*/"
      add-priority
         remote-label   FXTradingAdapter1
         remote-label   FXTradingAdapter2
      end-priority
   end-source-group
end-data-service

# Blotter channel (routed to Transformer first -- source affinity not used here)
add-data-service
   service-name         activity-blotter
   include-pattern      "^/PRIVATE/[^/]*/FX/BLOTTER/ACTIVITY"
   include-pattern      "^/PRIVATE/[^/]*/FX/BLOTTER/ITEM"
   add-source-group
      required true
      add-priority
         remote-label   Transformer1
         remote-label   Transformer2
      end-priority
   end-source-group
end-data-service

Transformer configuration

# Blotter channel
add-data-service
   service-name         activity-blotter
   include-pattern      "^/PRIVATE/[^/]*/FX/BLOTTER/ACTIVITY"
   include-pattern      "^/PRIVATE/[^/]*/FX/BLOTTER/ITEM"
   add-source-group
      required true
      affinity          fx-trading-adapters "/PRIVATE/([^-]+).*/"
      add-priority
         remote-label   FXTradingAdapter1
         remote-label   FXTradingAdapter2
      end-priority
   end-source-group
end-data-service

Activity blotter: two source-groups

This example demonstrates how to configure an activity blotter that records the combined activity of a trade adapter and an order adapter.

Configure the source groups of the blotter’s data service on Transformer with the same affinity keys as the source groups of their respective trade and order channels on Liberator.

Liberator configuration

# Object mapping for the trade channel
object-map   "/PRIVATE/TRADE/FX%1"   "/PRIVATE/%u/TRADE/FX%1"

# Object mapping for the orders channel
object-map   "/PRIVATE/ORDERS/FX%1"   "/PRIVATE/%u/ORDERS/FX%1"

# Object mapping for the activity blotter
object-map   "/PRIVATE/FX/BLOTTER/ACTIVITY%1"   "/PRIVATE/%u/FX/BLOTTER/ACTIVITY%1"
object-map   "/PRIVATE/FX/BLOTTER/ITEM/%1"   "/PRIVATE/%u/FX/BLOTTER/ITEM/%1"

# Trade Channel
add-data-service
   service-name         fx-trading
   include-pattern      "^/PRIVATE/[^/]*/TRADE/FX"
   add-source-group
      affinity          fx-trading-adapters "/PRIVATE/([^-]+).*/"
      required true
      add-priority
         remote-label   FXTradingAdapter1
         remote-label   FXTradingAdapter2
      end-priority
   end-source-group
end-data-service

# Order Channel
add-data-service
   service-name         fx-orders
   include-pattern      "^/PRIVATE/[^/]*/ORDERS/FX"
   add-source-group
      affinity          fx-order-adapters "/PRIVATE/([^-]+).*/"
      required true
      add-priority
         remote-label   FXOrdersAdapter1
         remote-label   FXOrdersAdapter2
      end-priority
   end-source-group
end-data-service

# Blotter channel (routed to Transformer first -- source affinity not used here)
add-data-service
   service-name         activity-blotter
   include-pattern      "^/PRIVATE/[^/]*/FX/BLOTTER/ACTIVITY"
   include-pattern      "^/PRIVATE/[^/]*/FX/BLOTTER/ITEM"
   add-source-group
      required true
      add-priority
         remote-label   Transformer1
         remote-label   Transformer2
      end-priority
   end-source-group
end-data-service

Transformer configuration

# Blotter channel
add-data-service
   service-name         activity-blotter
   include-pattern      "^/PRIVATE/[^/]*/FX/BLOTTER/ACTIVITY"
   include-pattern      "^/PRIVATE/[^/]*/FX/BLOTTER/ITEM"
   add-source-group
      affinity          fx-trading-adapters "/PRIVATE/([^-]+).*/"
      required true
      add-priority
         remote-label   FXTradingAdapter1
         remote-label   FXTradingAdapter2
      end-priority
   end-source-group
   add-source-group
      affinity          fx-order-adapters "/PRIVATE/([^-]+).*/"
      required true
      add-priority
         remote-label   FXOrdersAdapter1
         remote-label   FXOrdersAdapter2
      end-priority
   end-source-group
end-data-service

Reference: load-balancing and failover configurations

The table below details how source affinity works in various failover and load-balancing configurations.

— Do not use failover with source affinity for permissioning adapters. When using source affinity with permissioning adapters, all the permissioning adapters must be defined in one priority group (see scenario 3, below).
# Configuration Notes

1

 
add-source-group
    add-priority-group
        label FooAdapter1
    end-priority-group
    add-priority-group
        label FooAdapter2
    end-priority-group
end-source-group

Failover configuration with two adapters. Source affinity not enabled.

All subscriptions are routed to FooAdapter1.

If FooAdapter1 fails, then existing and new subscriptions will be routed to FooAdapter2. When FooAdapter1 recovers, existing subscriptions will continue to be served by FooAdapter2 and new subscriptions will be routed to FooAdapter1.

This configuration is not compatible with adapters that serve multiple channels per user. There is an edge case in which a user’s channels can be split across the two adapters. If there is a difference in the active-discard-timeout setting for the objects subscribed to by the channels, then if FooAdapter1 were to fail and recover, it is possible for a long-lived channel to remain subscribed to FooAdapter2 while all expired channels are reconnected to FooAdapter1 — splitting a user’s channels across the adapters and preventing their cooperation.

2

 
add-source-group
    affinity foo-adapters '/PRIVATE/([^-]+).*/'
    add-priority-group
        label FooAdapter1
    end-priority-group
    add-priority-group
        label FooAdapter2
    end-priority-group
end-source-group

Failover configuration with two adapters. Source affinity enabled.

All channels are routed to, and users given an affinity for, FooAdapter1.

If FooAdapter1 fails, then existing and new subscriptions will be routed to, and users given an affinity for, FooAdapter2. When FooAdapter1 recovers, subscriptions and affinities will not automatically be redistributed to FooAdapter1. Users with an affinity for FooAdapter2 will continue to see their subscriptions served by FooAdapter2 until one of the conditions below applies:

  • FooAdapter2 fails

  • the DataSource application (Liberator or Transformer) is restarted

  • all users with affinities under affinity key 'foo-adapters' log out.

Warning — This configuration is not compatible with permissioning adapters. Use scenario 3 instead.

3

 
add-source-group
    affinity foo-adapters '/PRIVATE/([^-]+).*/'
    add-priority-group
        label FooAdapter1
        label FooAdapter2
    end-priority-group
end-source-group

Load-balancing configuration with two adapters. Source affinity enabled.

How the load-balancer selects an adapter for a user’s subscription depends on whether the user already has a prior affinity for an adapter under the affinity key 'foo-adapters'.

  • If the user has a prior affinity for an adapter, the adapter will be selected to serve the new subscription too.

  • If the user does not have a prior affinity for an adapter, an adapter will be selected by the source affinity algorithm based on the user’s username, the affinity key and the number of adapters in the priority group. The algorithm will create an affinity between the user and the chosen adapter, stored under the affinity key 'foo-adapters'.

If FooAdapter1 or FooAdapter2 fails, then all existing subscriptions served by the failed adapter, and all affinities for the failed adapter, will be moved to the surviving adapter. New subscriptions, and their associated users' affinities, will be allocated to the surviving adapter. When the failed adapter recovers, subscriptions and affinities will not automatically be redistributed evenly across both adapters. Users with an affinity for the adapter that did not fail will continue to see their subscriptions served by that adapter until one of the conditions below applies:

  • the adapter that did not fail becomes unavailable

  • the DataSource application (Liberator or Transformer) is restarted

  • all users with affinities under affinity key 'foo-adapters' log out.

4

 
add-source-group
    affinity foo-adapters '/PRIVATE/([^-]+).*/'
    add-priority-group
        label FooAdapter1
        label FooAdapter2
    end-priority-group
    add-priority-group
        label FooAdapter3
        label FooAdapter4
    end-priority-group
end-source-group

Combined failover and load-balancing configuration with four adapters. Source affinity enabled.

While at least one adapter is available in the first priority group, the configuration will behave as configuration 2 does. When both adapters in the first priority group are unavailable, the configuration fails over to the second priority group.

On failure of the first priority group, existing subscriptions and affinities will be redistributed evenly across the adapters of the second priority group. The first priority group takes priority again when at least one adapter in the group recovers, but the subscriptions and affinities that were assigned to the second priority group will not return to the first priority group. Subscriptions and affinities assigned to the second priority group will continue to be honoured until one of the conditions below applies:

  • the adapters in the second priority group fail

  • the DataSource application (Liberator or Transformer) is restarted

  • all users with affinities under affinity key 'foo-adapters' log out.

Warning — This configuration is not compatible with permissioning adapters. Use scenario 3 instead.

See also: