DataSource peers configuration (part 1)

These DataSource configuration items define a DataSource application's connections to its DataSource peers.

Note: "this DataSource application" means the DataSource application for which you are defining the configuration. "DataSource peer" or "peer" means a DataSource application that this DataSource application communicates with.

More DataSource-peer configuration items can be found in DataSource peers configuration (part 2).

Contents:

add-peer

add-peer specifies a DataSource peer that this DataSource application can connect to. You can have a maximum of 1023 add-peer entries in your configuration.

Use in: C, Java (note not all options are available in Java)

Syntax:

add-peer
   local-label      [string]
   addr             [array of strings]
   port             [array of strings] 
   connect-timeout  [integer]
   heartbeat-slack-time [float]
   heartbeat-time   [integer]
   label            [string]
   local-flags      [integer/string]
   local-handshake-data​    [array of strings]
   local-id         [integer]
   local-label      [string]
   local-name       [string]
   local-type       [integer/string]
   monitor-interval [float]
   queue-size       [integer]
   remote-flags     [integer/string]
   remote-id        [integer]
   remote-label     [string]
   remote-name      [string]
   remote-type      [integer/string]
   request-timeout  [float]
   ssl                     [boolean]
   ssl-accept-certificate  [string]
   ssl-cipherlist          [string]
   ssl-passwordfile        [string]
   ssl-present-certificate [string]
   ssl-privatekey          [string]
   ssl-verify-mode         [string]
   thread-name      [string]
end-peer

The options for add-peer are:

Name Type Default Description
addr array of strings localhost

A space-separated list of peer IP addresses to connect to.

addr supports IPv6 addresses from version 7 of Caplin Platform.

Only include the addr and port options if this DataSource application initiates the connection to the peer, as opposed to listening for a connection from the peer. The peer must be configured to accept connections; this is done through the datasrc-port entry in the peer's configuration file.

If more than one address is provided, the additional addresses are used for failover in the situation where the connection to the first address fails. If this DataSource application can't connect to any of the addresses in the list, but it's trying to establish a connection for a data service, the application will try to fail over at the data service level (through multiple add-priority options within an add-source-group option of add-data-service).

connect-timeout integer

10 seconds

When this DataSource application requests to connect to a DataSource peer that either doesn't exist or to which there is currently no network route, the operating system will attempt to connect for a time that's dependent on various tunable operating system parameters - typically up to 4 minutes.

For the purposes of obtaining real-time data this timeout is too long; add-peer's connect-timeout option of add-peer allows you to change the timeout to a more suitable value in seconds. If the timeout expires, this DataSource application attempts to fail over to the next peer connection as defined in the addr option's list.

heartbeat-slack-time float 2.0 seconds

When this DataSource application doesn't receive an expected DataSource heartbeat from the peer, it waits heartbeat-slack-time seconds before disconnecting from the peer and trying to reconnect to it.

The minimum allowed value is 0.1 seconds.

Unlike heartbeat-time, the value of heartbeat-slack-time is not compared by peers.

For an explanation of heartbeats, see heartbeat-time.

heartbeat-time integer [disabled]

The two peers involved in a DataSource connection exchange heartbeat messages at regular intervals so that each of the DataSource applications can check that the other is still present, and take appropriate action if it is not (such as attempt to reconnect to the peer, or fail over to another peer).

heartbeat-time defines the time in seconds between each DataSource heartbeat. When the two peers have initially connected, they compare their heartbeat-time values and use the lowest.

The minimum allowed value is 1 second.

If you don't configure a heartbeat-time, heartbeats are disabled for the peer connection, unless enabled by the remote peer.

Also see heartbeat-slack-time, and How can I... Enable heartbeats between DataSource applications.

label string [none]

Deprecated. Use remote-label instead.

Note: This option isn't available in Java-based DataSource applications.

A label that uniquely defines (within the Caplin Platform installation) the DataSource peer.

This option is used in the add-data-service configuration item to refer to the peer connection over which data for the data service is to be obtained. Its value must match that of the datasrc-local-label configuration item in the peer's configuration, or it must match a local-label option in an add-peer item of the peer's configuration (where the add-peer item refers to this DataSource application).

Also see Note 1.

local-flags integer/
string
0 [= none]

Specifies flags determining the behaviour of this DataSource application when it starts up and establishes the connection to the peer, or when it reconnects to the peer.

Possible values are:

  • none or 0
    No special restart/reconnection behaviour.
  • recvautoreplay or 4
    When restarting, this DataSource application must accept replay updates from the peer. This flag is deprecated.

  • noconnect or 8
    When this DataSource application starts up, it must not automatically connect to the peer.

See also: remote-flags

local-handshake-data array of strings [Basic handshake data provided by the DataSource library - see Description]

A space-separated list of strings that are appended to the list of handshake data sent in the initial DataSource PEERINFO exchange with the DataSource peer defined by this add-peer item.

This option is only available in the following API versions:

The purpose of the handshake data is to provide version and configuration information to this DataSource application's peer. Basic handshake data is automatically appended to the PEERINFO message by the DataSource library that this application is using. This data consists of:

  • technology=<DataSource-implementation-type>,
  • datasrc-version=<DataSource-version-number>,
  • hostname=<hostname>,
  • platform=<platform-type>


For example, here's some handshake data created by a DataSource application implemented using the DataSource Java library:
technology=DSSJ, datasrc-version=6.2.5-dev, hostname=localhost, platform=x86-win32

Platform components, such as Liberator and Transformer, that are built using DataSource libraries, append their own configuration and version information to the handshake data.

You can use the local-handshake-data option to provide a DataSource peer with additional information about this DataSource application.

PEERINFO messages are logged in the DataSource application's event log and are also broadcast to JMX listeners (see DataSource monitoring and management).

local-id integer Value of datasrc-id

Deprecated. Use local-label instead.

An ID that uniquely defines this DataSource application within the Caplin Platform installation for the purposes of the connection to the peer defined by this add-peer entry. It overrides the setting of datasrc-id (if any).

You can use local-id to implement multiple connections to the same peer, typically for performance reasons. Just define an add-peer item for each connection and use a different local-id setting for each entry (for example, local-id 100 and local-id 101). In the peer's configuration, define an add-peer item that refers to this DataSource application, and add to it a remote-id option that refers to the relevant connection (for example, remote-id 100).

local-label

string

Value of datasrc-local-label

A label that uniquely defines this DataSource application within the Caplin Platform installation for the purposes of the connection to the peer defined by this add-peer entry. It overrides the setting of datasrc-local-label (if any). 

You can use local-label to implement multiple connections to the same peer, typically for performance reasons. Just define an add-peer item for each connection and use a different local-label setting for each entry (for example, local-label MyAppNameConnex1 and local-label MyAppNameConnex2). In the peer's configuration, define an add-peer item that refers to this DataSource application, and add to it a remote-label option that refers to the relevant connection (for example, remote-label MyAppNameConnex1).

You should use this option, in conjunction with remote-label, rather than the local-id option. (But, if you don't specify a local-label or datasrc-local-label, the DataSource application will fall back to identifying itself by a local-id if you've defined one, or by datasrc-id.)

Also see Note 1 and the example below.

local-name string

Value of datasrc-name

The name of this DataSource application by which the DataSource peer defined by this add-peer entry identifies this application. This overrides any name defined by the datasrc-name configuration item.

The name can contain the parameters %a and %h. At run time, %a is replaced by the DataSource application-name, and %h is replaced by the host name of the machine on which the DataSource application is running.

local-type integer/
string
0 (= none)

Specifies flags determining the type of this DataSource application as far as the peer is concerned. This information is sent to the peer when this DataSource application connects to the peer, and it overrides any remote-type option in the peer's configuration for the connection.

Possible values are:

  • none or broadcast or 0
    This DataSource application can only broadcast data. It can't accept subscription requests or contributions (data) from the peer.
  • active or  1
    This DataSource application is active - it can accept subscription requests for subjects from the peer, keep track of them, and send updates back to the peer for these subjects only (see Active subscription model).
  • contrib or  2
    This DataSource application can accept contributions (data) from the peer.

The flags can be ORed together using the '|' character; for example, local-type active|contrib means that this DataSource application can accept both subscription requests and contributions (data) from the peer.

Also see remote-type.

monitor-interval float Value of 
peer-monitor-interval

The interval in seconds at which statistics about this DataSource application's connection to the peer are read and transferred to the application's JMX monitoring module.

Note: This option isn't available in Java-based DataSource applications.
port array of integers [none]

A space-separated list of ports to connect to. The number of ports in the list must be the same as the number of addresses in the addr option.

Only include the addr and port options if this DataSource application initiates the connection to the peer, as opposed to listening for a connection from the peer. The peer must be configured to accept connections; this is done through the datasrc-port entry in the peer's configuration file.

If more than one port is provided the additional ports are used for failover in the situation where the connection to the first first address fails - see the explanation in the addr option.

queue-size integer 50 entries The maximum size of the queue to hold messages arriving from the peer.

remote-flags

integer/
string
0 (= none)

Specifies flags determining the behaviour of the peer when it starts up and establishes the connection to this DataSource application, or when it reconnects to this DataSource application. The values set by this option are overridden by the local-flags setting of the peer's configuration when the peer connection is made.

Possible values are:

  • none or 0
    No special restart/reconnection behaviour.
  • recvautoreplay or 4
    When restarting, the peer must accept replay updates from this DataSource application. This flag is deprecated.

  • noconnect or 8
    When the peer starts up, it must not automatically connect to this DataSource application.
remote-id integer 1

Deprecated.  Use remote-label instead.

An ID that uniquely defines (within the Caplin Platform installation) the DataSource peer.

This ID must match the value of datasrc-id defined in the peer's configuration (or the value of a local-id option in the peer's add-peer item that refers to this DataSource application).

remote-label string [none]

A label that uniquely defines (within the Caplin Platform installation) the DataSource peer. You should use this option, rather than the remote-id option. Its value must match that of the datasrc-local-label configuration item in the peer's configuration, or it must match a local-label option in an add-peer item of the peer's configuration (where the add-peer item refers to this DataSource application).

remote-label is also used in the add-priority configuration item to refer to the peer connection over which data for the data service is to be obtained.

Also see Note 1 and the example below.

remote-name string

src-n

The name of the DataSource peer. At run-time this name is replaced with the name defined in the peer's configuration (through datasrc-name or the local-name option of an add-peer item) when the peer connection is made.

The default value is src-n, where n is the index from zero of this add-peer entry in this DataSource application's configuration. For example, if this is the first add-peer entry in the configuration, the default remote-name is src-0, whereas if it's the second entry, the default value is src-1.

remote-type integer/
string
0 (= none)

Specifies flags determining the type of the peer. The values set by this option are overridden by the local-type setting of the peer's configuration when the peer connection is made.

Possible values are:

  • none or broadcast or 0
    The peer can only broadcast data. It can't accept subscription requests or contributions (data) from this DataSource application.
  • active or  1
    The peer is active - it can accept subscription requests for subjects from this DataSource application, keep track of them, and send updates back to this DataSource application for these subjects only (see Active subscription model).
  • contrib or  2
    The peer can accept contributions (data) sent to it from this DataSource application.

The flags can be ORed together using the '|' character; for example, remote-type active|contrib means that the peer can accept both subscription requests and contributions (data) from this DataSource application.

Also see Note 2.

request-timeout float Value of
source-request-timeout

The time in seconds for which this DataSource application waits for the peer to answer a request.  It overrides the global request timeout item source-request-timeout and is in turn overridden by the add-data-service timeout option, request-timeout.

Note: This option isn't available in Java-based DataSource applications.
ssl boolean FALSE

When ssl is set to TRUE this DataSource application can only communicate with the peer through a Secure Sockets Layer (SSL) connection:

  • If it initiates the connection to the peer, it establishes an SSL connection.
  • If it accepts connection requests from the peer, it can only an SSL type connection; any request for a non-secure connection is rejected.

In Java-based DataSource applications, you must explicitly set the option to TRUE:

Correct:

add-peer
   ssl TRUE
   ...
end-peer

Wrong:

add-peer
   ssl
   ...
end-peer

In C-based DataSource applications:

In Java-based DataSource applications, the certificate and private key are held in the Java KeyStore (see datasrc-ssl-keystore).

ssl-accept-certificate string [none]

When an SSL connection is being established with a peer, this option is used for certificate pinning: the DataSource application compares the public key in the certificate received from the peer against the public key specified by ssl-accept-certificate. If the keys don’t match, this application terminates the (probably unauthorised) connection.

(For more about certificate pinning, see the Wikipedia article at https://en.wikipedia.org/wiki/Certificate_pinning#Certificate_pinning.)

  • In C-based DataSource applications, ssl-accept-certificate specifies the path to, and filename of, an SSL certificate file that this DataSource application uses for certificate pinning. The file must be in PEM format.

    The directory path can contain the parameters %r and %a  At run time, %r is replaced by the root-directory (application-root) under which this DataSource application runs, and %a is replaced by the DataSource application-name.
  • In Java-based DataSource applicationsssl-accept-certificate specifies the name of an SSL certificate in the Java KeyStore (see datasrc-ssl-keystore); this DataSource application uses for this certificate for certificate pinning.

If you've set the ssl option to TRUE and haven't specified an ssl-accept-certificate or (for Java-based applications) an ssl-accept-certificate and  datasrc-ssl-keystore, the DataSource application terminates in error.

ssl-cipherlist string Value of datasrc-ssl-cipherlist

A list of SSL ciphers that the peer connection can use. Overrides the cipher list defined by datasrc-ssl-cipherlist.

For syntax and possible values, see datasrc-ssl-cipherlist.

ssl-passwordfile

 

string [none] (= no password needed)

The path to, and filename of, a password file that contains a plain text (unencrypted) password used to access private key files and certificate files.

In a C-based DataSource application, the password provides access to the password-protected private key file defined by the ssl-privatekey option and access to the certificate file (ssl-present-certificate) that's paired with this private key.

Note: This option isn't available in Java-based DataSource applications; use datasrc-ssl-passwordfile instead.
Important: We strongly recommend that you password protect your private key files, certificate files and Java KeyStores.
ssl-present-certificate string [none]

Specifies the SSL certificate file that this DataSource application sends to the peer when initiating an SSL connection with it. This is the certificate that identifies this DataSource application to the peer.

  • In C-based DataSource applications, ssl-present-certificate specifies the path to, and filename of, the SSL certificate file. The file must be in PEM format.

    The directory path can contain the parameters %r and %a At run time, %r is replaced by the root-directory (application-root) under which this DataSource application runs, and %a is replaced by the DataSource application-name.
  • In Java-based DataSource applications, ssl-present-certificate specifies the name of an SSL certificate in the Java KeyStore (see datasrc-ssl-keystore).

Specify this option when the ssl option is set to TRUE and this DataSource application initiates the connection to the peer - if you don't provide a valid ssl-present-certificate, the DataSource application terminates in error.

When this DataSource application acts as an SSL server (that is, it accepts incoming SSL connections from peers), the certificate that it sends to the connecting peer is the one defined by datasrc-ssl-certificate.
ssl-privatekey string [none]

Specifies the path to, and filename of, the SSL private key associated with the certificate specified by ssl-present-certificate. This DataSource application uses the key to decrypt the symmetric session key received from the peer. The private key file must be in PEM format.

Specify this option when the ssl option is set to TRUE and this DataSource application initiates the connection to the peer  - if you don't provide a valid ssl-privatekey, the DataSource application terminates in error.

The directory path can contain the parameters %r and %a  At run time, %r is replaced by the root-directory (application-root) under which this DataSource application runs, and %a is replaced by the DataSource application-name.

Note: This option isn't available in Java-based DataSource applications. In these applications, the private key is held in the Java KeyStore (see datasrc-ssl-keystore).
When this DataSource application acts as an SSL server (that is, it accepts incoming SSL connections from peers), the private key that it uses is the one defined by datasrc-ssl-privatekey.
ssl-verify-mode string SSL_VERIFY_FAIL_
IF_NO_PEER_CERT|
SSL_VERIFY_PEER

Specifies how this SSL connection behaves during the SSL handshake.

Note: This option isn't available in Java-based DataSource applications.

Values accepted are:

  • SSL_VERIFY_FAIL_IF_NO_PEER_CERT
  • SSL_VERIFY_PEER
  • SSL_VERIFY_NONE
  • SSL_VERIFY_CLIENT_ONCE

​The meanings of these settings are given in the NOTES section of the OpenSSL documentation for SSL_CTX_set_verify(3).

You can combine values together using the | operator, like this:

ssl-verify-mode SSL_VERIFY_PEER|SSL_VERIFY_CLIENT_ONCE

thread-name string An empty string

Defines a named thread on which communications with the peer are processed. If the same thread name is defined for more than one DataSource peer, all these peers share this same thread.

Note: This option isn't available in Java-based DataSource applications.

If this option is not defined, or is an empty string, but the global configuration option peer-thread-pool-size is more than zero, a thread from the global pool of peer connection threads is allocated to the peer.  If neither thread-name nor peer-thread-pool-size is defined, a dedicated thread is allocated to the peer.

thread-name can be a maximum of 15 characters long. It is recommended that the first 10 characters be unique within this DataSource application, so that peer connection thread names can be easily distinguished when appearing in the application's logs and the output of Linux commands such as top.

thread-name must not be set to a string beginning with “peer” or “ptpt”, as these prefixes are used for internally generated thread names.

For more about using thread-name, see How can I... Configure threads for peer communication.

Note 1: Platform blades, and Liberator and Transformer that run under version the Caplin Platform Deployment Framework use the deprecated label option in their configuration, rather than remote-label. This will change in a future release of the Framework.

Note 2: The setting of the remote-type option can appear in log files and is available to monitoring utilities. In addition, if a client application requests data that's provided by a DataSource peer of this type, but the remote DataSource peer has not yet connected to the Liberator, Liberator uses this information to inform the client application that the service is not available because the DataSource peer is down.

Example of datasrc-local-label and remote-label

This example shows how the datasrc-local-label configuration item and and the remote-label option of add-peer are used to define the connections between DataSource applications.

Assume you have two DataSource applications, one is a Liberator called MyLiberator and the other is an Integration Adapter called MyIntegrationAdapter. The following configurations allow MyIntegrationAdapter to connect to MyLiberator. There's also a data service defined for MyLiberator (using add-data-service) that enables the Liberator to subscribe to FX prices supplied by MyIntegrationAdapter.

Peer connnection configuration for MyIntegrationAdapter:

datasrc-name MyIntegrationAdapter-%h
datasrc-local-label MyIntegrationAdapter

add-peer
   remote-name MyLiberator
   remote-label MyLiberator
   addr         <MyLiberators-hostname>
   port         <MyLiberators-portnumber>
end-peer

Peer connection configuration for MyLiberator:

datasrc-name MyLiberator-%h
datasrc-local-label MyLiberator

add-peer
   remote-name MyIntegrationAdapter
   remote-label MyIntegrationAdapter
end-peer

add-data-service
   service-name    fx-prices
   include-pattern ^/FX/
   add-source-group
      required   TRUE
      retry-time 20
      add-priority
         remote-label MyIntegrationAdapter
      end-priority
   end-source-group 
end-data-service

See also: