DataSource peers configuration (part 1)

These DataSource configuration items define a DataSource application’s connections to its DataSource peers.

"this DataSource application" means the DataSource application for which you are defining the configuration. "DataSource peer" or "peer" means a DataSource application that this DataSource application communicates with.

More DataSource-peer configuration items can be found in DataSource peers configuration (part 2).

add-peer

add-peer specifies a DataSource peer that this DataSource application can connect to. You can have a maximum of 1023 add-peer entries in your configuration.

Use in: C, Java (note not all options are available in Java)

Syntax
add-peer
   addr                    [array of strings]
   apikey                  [string]
   connect-timeout         [integer]
   heartbeat-slack-time    [float]
   heartbeat-time          [integer]
   label                   [string]
   local-flags             [integer/string]
   local-handshake-data    [array of strings]
   local-id                [integer]
   local-label             [string]
   local-name              [string]
   local-type              [integer/string]
   monitor-interval        [float]
   port                    [array of strings]
   queue-size              [integer]
   remote-flags            [integer/string]
   remote-id               [integer]
   remote-label            [string]
   remote-name             [string]
   remote-type             [integer/string]
   request-timeout         [float]
   ssl                     [boolean]
   ssl-accept-certificate  [string]
   ssl-cipherlist          [string]
   ssl-passwordfile        [string]
   ssl-present-certificate [string]
   ssl-privatekey          [string]
   ssl-verify-mode         [string]
   thread-name             [string]
   websocket               [boolean]
   websocket-path          [string]
end-peer
Options
addr

Type: array of strings | Default: localhost

addr supports IPv6 addresses from version 7 of Caplin Platform.

Only include the addr and port options if this DataSource application initiates the connection to the peer, as opposed to listening for a connection from the peer. The peer must be configured to accept connections; this is done through the datasrc-port entry, or the datasrc-ws-port entry when websocket is TRUE, in the peer’s configuration file.

If more than one address is provided, the additional addresses are used for failover in the situation where the connection to the first address fails. If this DataSource application can’t connect to any of the addresses in the list, but it’s trying to establish a connection for a data service, the application will try to fail over at the data service level (through multiple add-priority options within an add-source-group option of add-data-service).

apikey

Type: string | Default: null | Since: DataSource 8

Specifies the API key to use for this connection. Required if the remote peer has one or more API keys configured with add-apikey and the connection uses one or more of the following features: Dynamic Peers, Dynamic Services, and Dynamic Fields.

connect-timeout

Type: integer | Default: 10

When this DataSource application requests to connect to a DataSource peer that either doesn’t exist or to which there is currently no network route, the operating system will attempt to connect for a time that’s dependent on various tunable operating system parameters - typically up to 4 minutes.

For the purposes of obtaining real-time data this timeout is too long; add-peer's connect-timeout option of add-peer allows you to change the timeout to a more suitable value in seconds. If the timeout expires, this DataSource application attempts to fail over to the next peer connection as defined in the addr option’s list.

heartbeat-slack-time

Type: float | Default: 2.0 (7.1.0-7.1.15), value of datasrc-heartbeat-slack-time (7.1.16+)

When this DataSource application doesn’t receive an expected DataSource heartbeat from the peer, it waits heartbeat-slack-time seconds before disconnecting from the peer and trying to reconnect to it.

The minimum allowed value is 0.1 seconds.

After modifying any of the heartbeat configuration items, restart both the DataSource application and its peers for the changes to take effect.

Unlike heartbeat-time, the value of heartbeat-slack-time is not compared by peers.

For an explanation of heartbeats, see heartbeat-time.

heartbeat-time

Type: integer | Default: null (7.1.0-7.1.15), value of datasrc-heartbeat-time (7.1.16+)

The two peers involved in a DataSource connection exchange heartbeat messages at regular intervals so that each of the DataSource applications can check that the other is still present, and take appropriate action if it is not (such as attempt to reconnect to the peer, or fail over to another peer).

heartbeat-time defines the time in seconds between each DataSource heartbeat. When two DataSource applications connect, they compare their configured heartbeat times and use the lowest.

The minimum allowed value is 1 second.

After modifying any of the heartbeat configuration items, restart both the DataSource application and its peers for the changes to take effect.

label

Type: string | Default: null | Deprecated: use remote-label instead

This option isn’t available in Java-based DataSource applications.

A label that uniquely defines (within the Caplin Platform installation) the DataSource peer.

This option is used in the add-data-service configuration item to refer to the peer connection over which data for the data service is to be obtained. Its value must match that of the datasrc-local-label configuration item in the peer’s configuration, or it must match a local-label option in an add-peer item of the peer’s configuration (where the add-peer item refers to this DataSource application).

Also see Note 1.

local-flags

Type: integer/string | Default: 0

Specifies flags determining the behaviour of this DataSource application when it starts up and establishes the connection to the peer, or when it reconnects to the peer.

Overrides datasrc-local-flags for this peer connection.

Possible values are:

  • none or 0: No special restart/reconnection behaviour.

  • recvautoreplay or 4: When restarting, this DataSource application must accept replay updates from the peer. This flag is deprecated.

  • noconnect or 8: When this DataSource application starts up, it must not automatically connect to the peer.

  • nofields: Do not send Dynamic Fields data in the DataSource handshake. Required for a DataSource 8 application to connect to Liberator/DSDK < 7.1.31 or Transformer < 7.1.19.

See also: remote-flags

local-handshake-data

Type: array of strings | Default: basic handshake data provided by the DataSource library

A space-separated list of strings that are appended to the list of handshake data sent in the initial DataSource PEERINFO exchange with the DataSource peer defined by this add-peer item.

This option is only available in the following API versions:

The purpose of the handshake data is to provide version and configuration information to this DataSource application’s peer. Basic handshake data is automatically appended to the PEERINFO message by the DataSource library that this application is using. This data consists of:

  • technology=<DataSource-implementation-type>,

  • datasrc-version=<DataSource-version-number>,

  • hostname=<hostname>,

  • platform=<platform-type>

For example, here’s some handshake data created by a DataSource application implemented using the DataSource Java library:

technology=DSSJ, datasrc-version=6.2.5-dev, hostname=localhost, platform=x86-win32

Platform components, such as Liberator and Transformer, that are built using DataSource libraries, append their own configuration and version information to the handshake data.

You can use the local-handshake-data option to provide a DataSource peer with additional information about this DataSource application.

PEERINFO messages are logged in the DataSource application’s event log and are also broadcast to JMX listeners (see DataSource monitoring and management).

local-id

Type: integer | Default: value of datasrc-id | Discouraged: use local-label

This option will be removed in a later version and its use is discouraged. Use local-label instead.

An ID that uniquely defines this DataSource application within the Caplin Platform installation for the purposes of the connection to the peer defined by this add-peer entry. It overrides the setting of datasrc-id (if any).

You can use local-id to implement multiple connections to the same peer, typically for performance reasons. Just define an add-peer item for each connection and use a different local-id setting for each entry (for example, local-id 100 and local-id 101). In the peer’s configuration, define an add-peer item that refers to this DataSource application, and add to it a remote-id option that refers to the relevant connection (for example, remote-id 100).

local-label

Type: string | Default: value of datasrc-local-label

A label that uniquely defines this DataSource application within the Caplin Platform installation for the purposes of the connection to the peer defined by this add-peer entry. It overrides the setting of datasrc-local-label (if any).

You can use local-label to implement multiple connections to the same peer, typically for performance reasons. Just define an add-peer item for each connection and use a different local-label setting for each entry (for example, local-label MyAppNameConnex1 and local-label MyAppNameConnex2). In the peer’s configuration, define an add-peer item that refers to this DataSource application, and add to it a remote-label option that refers to the relevant connection (for example, remote-label MyAppNameConnex1).

You should use this option, in conjunction with remote-label, rather than the local-id option. (But, if you don’t specify a local-label or datasrc-local-label, the DataSource application will fall back to identifying itself by a local-id if you’ve defined one, or by datasrc-id.)

Also see Note 1 and the example below.

local-name

Type: string | Default: value of datasrc-name | Discouraged: use local-label

The name of this DataSource application by which the DataSource peer defined by this add-peer entry identifies this application. This overrides any name defined by the datasrc-name configuration item.

The name can contain the parameters %a and %h. At run time, %a is replaced by the DataSource application-name, and %h is replaced by the host name of the machine on which the DataSource application is running.

local-type

Type: integer/string | Default: 0

Specifies flags determining the type of this DataSource application as far as the peer is concerned. This information is sent to the peer when this DataSource application connects to the peer, and it overrides any remote-type option in the peer’s configuration for the connection.

Possible values are:

  • none or broadcast or 0

    This DataSource application can only broadcast data. It can’t accept subscription requests or contributions (data) from the peer.

  • active or 1

    This DataSource application is active - it can accept subscription requests for subjects from the peer, keep track of them, and send updates back to the peer for these subjects only (see Active subscription model).

  • contrib or 2

    This DataSource application can accept contributions (data) from the peer.

The flags can be ORed together using the ‘|’ character; for example, local-type active|contrib means that this DataSource application can accept both subscription requests and contributions (data) from the peer.

Also see remote-type.

monitor-interval

Type: float | Default: value of peer-monitor-interval

The interval in seconds at which statistics about this DataSource application’s connection to the peer are read and transferred to the application’s JMX monitoring module.

This option isn’t available in Java-based DataSource applications.
port

Type: array of integers | Default: null

A space-separated list of ports to connect to. The number of ports in the list must be the same as the number of addresses in the addr option.

Only include the addr and port options if this DataSource application initiates the connection to the peer, as opposed to listening for a connection from the peer. The peer must be configured to accept connections; this is done through the datasrc-port entry, or the datasrc-ws-port entry when websocket is TRUE, in the peer’s configuration file.

If more than one port is provided the additional ports are used for failover in the situation where the connection to the first first address fails - see the explanation in the addr option.

queue-size

Type: integer | Default: 50

The maximum size of the queue to hold messages arriving from the peer.

remote-flags

Type: integer/string | Default: 0

Specifies flags determining the behaviour of the peer when it starts up and establishes the connection to this DataSource application, or when it reconnects to this DataSource application. The values set by this option are overridden by the local-flags setting of the peer’s configuration when the peer connection is made.

Possible values are:

  • none or 0

    No special restart/reconnection behaviour.

  • recvautoreplay or 4

    When restarting, the peer must accept replay updates from this DataSource application. This flag is deprecated.

  • noconnect or 8

    When the peer starts up, it must not automatically connect to this DataSource application.

remote-id

Type: integer | Default: 1 | Discouraged: use remote-label

This option will be removed in a later version and its use is discouraged. Use remote-label instead.

An ID that uniquely defines (within the Caplin Platform installation) the DataSource peer.

This ID must match the value of datasrc-id defined in the peer’s configuration (or the value of a local-id option in the peer’s add-peer item that refers to this DataSource application).

remote-label

Type: string | Default: null

A label that uniquely defines (within the Caplin Platform installation) the DataSource peer. You should use this option, rather than the remote-id option. Its value must match that of the datasrc-local-label configuration item in the peer’s configuration, or it must match a local-label option in an add-peer item of the peer’s configuration (where the add-peer item refers to this DataSource application).

remote-label is also used in the add-priority configuration item to refer to the peer connection over which data for the data service is to be obtained.

Also see Note 1 and the example below.

remote-name

Type: string | Default: src-n | Discouraged: use remote-label

The name of the DataSource peer. At run-time this name is replaced with the name defined in the peer’s configuration (through datasrc-name or the local-name option of an add-peer item) when the peer connection is made.

The default value is src-n, where n is the index from zero of this add-peer entry in this DataSource application’s configuration. For example, if this is the first add-peer entry in the configuration, the default remote-name is src-0, whereas if it’s the second entry, the default value is src-1.

remote-type

Type: integer/string | Default: 0

Specifies flags determining the type of the peer. The values set by this option are overridden by the local-type setting of the peer’s configuration when the peer connection is made.

Possible values are:

  • none or broadcast or 0

    The peer can only broadcast data. It can’t accept subscription requests or contributions (data) from this DataSource application.

  • active or 1

    The peer is active - it can accept subscription requests for subjects from this DataSource application, keep track of them, and send updates back to this DataSource application for these subjects only (see Active subscription model).

  • contrib or 2

    The peer can accept contributions (data) sent to it from this DataSource application.

The flags can be ORed together using the ‘|’ character; for example, remote-type active|contrib means that the peer can accept both subscription requests and contributions (data) from this DataSource application.

Also see Note 2.

request-timeout

Type: float | Default: value of source-request-timeout

The time in seconds this DataSource application waits for the peer to answer a request.

Peer-request timeout precedence (highest first): add-peer::request-timeout > peer-request-timeout > source-request-timeout.

A request to a peer in the context of a data-service is also subject to the data-service’s request timeout. For more information, see add-data-service::request-timeout and service-request-timeout.

This option isn’t available in Java-based DataSource applications.
ssl

Type: boolean | Default: FALSE

When ssl is set to TRUE, DataSource connections between this DataSource application and the peer must be over SSL. If this application initiates DataSource connections to the peer, then the connections will be initiated over SSL. If this application accepts DataSource connections from the peer, then only connections over SSL will be accepted.

In C DataSource applications, specifying ssl is enough to enable SSL. In Java DataSource applications, you must specify the boolean value explicitly: ssl TRUE.

Overrides global configuration item datasrc-ssl-enable for this peer connection.

When ssl is set to TRUE, the following configuration options are required in this DataSource and the peer:

ssl-accept-certificate

Type: string array | Default: null

When TLS connections between peers are enabled, you must specify an 'accept' certificate (this option or datasrc-accept-ssl-certificate) and a 'present' certificate (ssl-present-certificate or datasrc-present-ssl-certificate). If you fail to do so, the DataSource fails to start.

Overrides the global configuration item datasrc-accept-ssl-certificate for this peer connection.

Specifies one or more TLS certificates accepted (trusted) by this DataSource. This option is used for certificate pinning. The peer must present an accepted certificate or the TLS handshake fails.

  • DataSource C: ssl-accept-certificate specifies the path to, and filename of, an SSL certificate file (PEM) that this DataSource application uses for certificate pinning.

    The directory path can contain the parameters %r and %a. At run time, %r is replaced by the root-directory (application-root) under which this DataSource application runs, and %a is replaced by the DataSource application-name.

  • DataSource Java: ssl-accept-certificate specifies the name of an SSL certificate in the Java KeyStore (see datasrc-ssl-keystore) that this DataSource application uses for certificate pinning.

If you’ve set the ssl option to TRUE and haven’t specified an ssl-accept-certificate or (for Java-based applications) an ssl-accept-certificate and datasrc-ssl-keystore, the DataSource application terminates in error.

ssl-cipherlist

Type: string | Default: value of datasrc-ssl-cipherlist

A list of SSL ciphers that the peer connection can use. Overrides the cipher list defined by datasrc-ssl-cipherlist.

For syntax and possible values, see datasrc-ssl-cipherlist.

ssl-passwordfile

Type: string | Default: null

The path to, and filename of, a password file that contains a plain text (unencrypted) password used to access private key files and certificate files.

When a value for this option is not specified, the DataSource assumes that no password is required to access private key files and certificate files.

In a C-based DataSource application, the password provides access to the password-protected private key file defined by the ssl-privatekey option and access to the certificate file (ssl-present-certificate) that’s paired with this private key.

This option isn’t available in Java-based DataSource applications; use datasrc-ssl-passwordfile instead.
We strongly recommend that you password protect your private key files, certificate files and Java KeyStores.
ssl-present-certificate

Type: string | Default: null

When TLS connections between peers are enabled, you must specify an 'accept' certificate (ssl-accept-certificate or datasrc-accept-ssl-certificate) and a 'present' certificate (this option or datasrc-present-ssl-certificate). If you fail to do so, the DataSource fails to start.

Overrides the global configuration item datasrc-present-ssl-certificate for this peer connection.

Specifies the TLS certificate that identifies this DataSource:

  • DataSource C: ssl-present-certificate specifies the path to, and filename of, the SSL certificate file (PEM).

    The directory path can contain the parameters %r and %a At run time, %r is replaced by the root-directory (application-root) under which this DataSource application runs, and %a is replaced by the DataSource application-name.

  • DataSource Java: ssl-present-certificate specifies the name of an SSL certificate in the Java KeyStore (see datasrc-ssl-keystore).

Specify this option when the ssl option is set to TRUE and this DataSource application initiates the connection to the peer - if you don’t provide a valid ssl-present-certificate, the DataSource application terminates in error.

ssl-privatekey

Type: string | Default: null

Specifies the path to, and filename of, the SSL private key associated with the certificate specified by ssl-present-certificate. This DataSource application uses the key to decrypt the symmetric session key received from the peer. The private key file must be in PEM format.

For this peer connection, ssl-present-certificate overrides the value defined by datasrc-ssl-present-certificate.

Specify this option when the ssl option is set to TRUE and this DataSource application initiates the connection to the peer - if you don’t provide a valid ssl-privatekey, the DataSource application terminates in error.

The directory path can contain the parameters %r and %a At run time, %r is replaced by the root-directory (application-root) under which this DataSource application runs, and %a is replaced by the DataSource application-name.

This option isn’t available in Java-based DataSource applications. In these applications, the private key is held in the Java KeyStore (see datasrc-ssl-keystore).
When this DataSource application acts as an SSL server (that is, it accepts incoming SSL connections from peers), the private key that it uses is the one defined by datasrc-ssl-privatekey.
ssl-verify-mode

Type: string | Default: SSL_VERIFY_FAIL_IF_NO_PEER_CERT|SSL_VERIFY_PEER

Specifies how this SSL connection behaves during the SSL handshake.

This option isn’t available in Java-based DataSource applications.

Values accepted are:

  • SSL_VERIFY_FAIL_IF_NO_PEER_CERT

  • SSL_VERIFY_PEER

  • SSL_VERIFY_NONE

  • SSL_VERIFY_CLIENT_ONCE

  • SSL_VERIFY_POST_HANDSHAKE (from Liberator 7.1.5)

The meanings of these settings are given in the NOTES section of the OpenSSL documentation for SSL_CTX_set_verify(3).

You can combine values together using the | operator, like this:

ssl-verify-mode SSL_VERIFY_PEER|SSL_VERIFY_CLIENT_ONCE

thread-name

Type: string | Default: empty string

Defines a named thread on which communications with the peer are processed. If the same thread name is defined for more than one DataSource peer, all these peers share this same thread.

This option isn’t available in Java-based DataSource applications.

If this option is not defined, or is an empty string, but the global configuration option peer-thread-pool-size is more than zero, a thread from the global pool of peer connection threads is allocated to the peer. If neither thread-name nor peer-thread-pool-size is defined, a dedicated thread is allocated to the peer.

thread-name can be a maximum of 15 characters long. It is recommended that the first 10 characters be unique within this DataSource application, so that peer connection thread names can be easily distinguished when appearing in the application’s logs and the output of Linux commands such as top.

thread-name must not be set to a string beginning with “peer” or “ptpt”, as these prefixes are used for internally generated thread names.

For more about using thread-name, see How can I…​ Configure threads for peer communication.

websocket

Type: boolean | Default: FALSE | Since: DSDK 7.1.30 (Liberator 7.1.30, Transformer 7.1.18) and DataSource for Java 7.1.22.

When websocket is set to TRUE, DataSource connections between this DataSource application and the peer will be made over the WebSocket protocol.

If this add-peer configuration includes the addr and port options, this DataSource application will attempt to make a WebSocket connection to the peer at the address, port, and URI specified by the addr, port, and websocket-path options respectively.

If this add-peer configuration does not include both the addr and port options, the DataSource application will expect a WebSocket connection from the peer on the port and URI specified by this DataSource application’s datasrc-ws-port and datasrc-ws-path configuration items respectively.

websocket-path

Type: string | Default: / | Since: DSDK 7.1.30 (Liberator 7.1.30, Transformer 7.1.18) and DataSource for Java 7.1.22.

The URI of the peer’s WebSocket service at addr and port.

The port and URI of the peer’s WebSocket service is set on the peer by the configuration items datasrc-ws-port and datasrc-ws-path respectively.

If this add-peer configuration item does not include both addr and port options, or the websocket option is set to FALSE, then websocket-path has no effect.

Note 1: Platform blades, and Liberator and Transformer that run under version the Caplin Platform Deployment Framework use the deprecated label option in their configuration, rather than remote-label. This will change in a future release of the Framework.

Note 2: The setting of the remote-type option can appear in log files and is available to monitoring utilities. In addition, if a client application requests data that’s provided by a DataSource peer of this type, but the remote DataSource peer has not yet connected to the Liberator, Liberator uses this information to inform the client application that the service is not available because the DataSource peer is down.

Example of datasrc-local-label and remote-label

This example shows how the datasrc-local-label configuration item and the remote-label option of add-peer are used to define the connections between DataSource applications.

Assume you have two DataSource applications, one is a Liberator called MyLiberator and the other is an Integration Adapter called MyIntegrationAdapter. The following configurations allow MyIntegrationAdapter to connect to MyLiberator. There’s also a data service defined for MyLiberator (using add-data-service) that enables the Liberator to subscribe to FX prices supplied by MyIntegrationAdapter.

Peer connnection configuration for MyIntegrationAdapter
datasrc-name MyIntegrationAdapter-%h
datasrc-local-label MyIntegrationAdapter

add-peer
   remote-name MyLiberator
   remote-label MyLiberator
   addr         <MyLiberators-hostname>
   port         <MyLiberators-portnumber>
end-peer
Peer connection configuration for MyLiberator
datasrc-name MyLiberator-%h
datasrc-local-label MyLiberator

add-peer
   remote-name MyIntegrationAdapter
   remote-label MyIntegrationAdapter
end-peer

add-data-service
   service-name    fx-prices
   include-pattern ^/FX/
   add-source-group
      required   TRUE
      retry-time 20
      add-priority
         remote-label MyIntegrationAdapter
      end-priority
   end-source-group
end-data-service

See also: