DataSource configuration syntax

This page describes the configuration syntax used by C DataSource API applications from version 5 and Java DataSource API applications from version 6.

Earlier DataSource API versions used an XML configuration format that was deprecated in the DataSource 6 APIs and removed in the DataSource 7 APIs.

Introduction

DataSource applications can be configured by settings stored in one or more configuration files, or by settings stored in a database and supplied to the DataSource application by a configuration script.

This document describes:

  • The syntax of the different types of configuration item used to configure a DataSource application.

  • How variables, conditionals, and macros can be used in a configuration file.

  • How configuration settings supplied by a web server or configuration script can be used to configure a DataSource application.

For a description of the configuration items that you can use to configure a particular DataSource application (such as Caplin Liberator, Caplin Transformer, or an Integration Adapter), refer to the Administration Guide for the application that you want to configure.

For a description of the DataSource communications infrastructure and how it links Caplin components together, see Caplin Platform > DataSource.

Types of configuration item

Four different types of configuration item can be used to configure a DataSource application:

Some of these configuration items allow you to specify a list of one or more values (see Lists).

Boolean configuration items

A boolean configuration item does not take any value, but is set to true if present in the DataSource configuration, and to false (the default) if omitted from the configuration.

An example of a boolean configuration item is latency-chain-enable, which enables or disables latency chaining.

Example (C DataSource API)
latency-chain-enable
Example (Java DataSource API)
latency-chain-enable TRUE

Single-parameter configuration items

Single parameter configuration items take one parameter:

configuration item literal ${ configuration variable } ${ENV: environment variable } ${EVAL: expression }

The parameter can be assigned a literal, a configuration variable, an environment variable, or an expression.

An example of this type of configuration item is datasrc-port, which specifies the network port that the application listens on for DataSource messages and connection requests.

datasrc-port 22001

In this case the listening port is set to 22001.

If present in the configuration file, a single value configuration item must define the value to be assigned. If not present in the configuration file, the configuration item is assigned a default value.

Multiple-parameter configuration items

Multiple-parameter configuration items take two or more parameters:

configuration item literal ${ configuration variable } ${ENV: environment variable } ${EVAL: expression }

Each parameter can be assigned a literal, a configuration variable, an environment variable, or an expression.

An example of this type of configuration item is add-field, which maps a field name (a string) to a field number (an integer):

add-field Bid 22

In this case the Bid field is mapped to field number 22.

If a multiple-parameter configuration item is present in the configuration file, values must be defined for all parameters that are not optional (such as the field name and field number in the example above).

If a parameter is described as optional, the value for that related item can be omitted from the configuration. The parameter will be assigned its default value.

Values are assigned to parameters according to the position of the value in the list of ordered arguments.

If the configuration item is not present in the configuration file, related items are assigned default values.

Configuration items with nested options

This type of configuration item takes nested options, where each nested option can be any one of the four types of configuration item (boolean, single parameter, multiple parameter, and nested).

An example of this type of configuration item is add-peer, which specifies options for connecting to a remote DataSource application (see Data services).

add-peer
  addr liberator.example.com
  port 25000
end-peer

In this case the address of the remote DataSource is set to liberator.example.com, and the port that it listens on to 25000.

Lists

Some configuration items and options allow you to specify a list of one or more values, where each value is the same type.

An example is addr, a nested option of add-peer, which defines a list of addresses that the DataSource application will attempt to connect to:

addr 192.168.1.1 192.168.1.2 192.168.1.3

In the reference sections of the DataSource documentation, the syntax definition specifies the types that can be listed, as shown in the following table.

List Type Description Example

string list

A space separated list (array) of one or more strings.

A space separated list of IP addresses: 127.0.0.0 192.255.129.1

integer list

A space separated list (array) of one or more integers.

A space separated list of port numbers: 22001 22002

Literals

Literal values in a configuration file can be numbers or strings.

If a literal value contains one or more spaces, it must be enclosed in single or double quotes. To escape a quote character in a literal value, precede it with a backslash (\).

Examples:

Setting the application name to the literal 'TradingAdapter' (no quotes required)
application-name TradingAdapter
Setting the application name to the literal 'Trading Adapter' (quotes required)
application-name "Trading Adapter"

Configuration variables

This section covers defining, undefining, and referencing custom variables in configuration files.

Defining a variable

You define a configuration variable using the define configuration directive:

define VARIABLE literal ${ configuration variable } ${ENV: environment variable } ${EVAL: expression }

You can set a configuration variable to a literal, a configuration variable, an environment variable, or an expression.

By convention, variable names in Caplin configuration files are in upper case.

The scope of the variable is the configuration file in which it is defined, and any files that include this configuration file using the include-file directive (see Including multiple configuration files).

For example, to define the variable HTTP_PORT with the value 8080, use the configuration below:

define HTTP_PORT 8080

Undefining a variable

You undefine a configuration variable using the undefine configuration directive:

undefine VARIABLE

For example, to undefine the variable HTTP_PORT, use the configuration below:

undefine HTTP_PORT

Referencing a variable

The following notation shows how a variable is used to set the value of a configuration item.

${ VARIABLE }

When a variable is used to set the value of a configuration item, the name of the variable must be enclosed in curly braces and preceded by the $ character. If an undefined variable is used in the configuration, the value of the configuration item is set to an empty string and a warning is logged to the console and log file.

The following example defines a variable, and sets the value of a configuration item using that variable.

Example: defining and using a variable
define HTTP_PORT 8080
http-port ${HTTP_PORT}

In this case, the value of Liberator’s http-port configuration item is set to 8080 (using the HTTP_PORT variable).

Variables can also be concatenated with surrounding text and other variables.

Example: concatenating a variable with surrounding text
define HTTP_PORT 8080
http-port 1${HTTP_PORT}

In this case, the value of Liberator’s http-port configuration item is set to 18080.

Predefined variables

The following variables are predefined by DataSource libraries:

C DataSource API and .NET DataSource API predefined variables
Variable Description

APP_NAME

The name of the DataSource application, as defined by application-name.

ccd

Current configuration directory. The full path to the directory containing the configuration file.

HOME_DIR

The home directory of the process owner.

HOST_NAME

The name of the host machine

MAJOR_VERSION

The major version of the DataSource library (for example, 7)

MINOR_VERSION

The minor version of the DataSource library (for example, 0)

OS

The name of the operating system that the DataSource library was built for. Examples are win64 and linux-EL6-gnu.

OS_TRIPLET

The GNU configuration triplet (of the form cpu-manufacturer-operating_system). Examples are x86_64-pc-win64 and x86_64-pc-linux-EL6-gnu.

Java DataSource API predefined variables
Variable Description

application

The name of the DataSource application, as defined by application-name.

ccd

Current configuration directory. The full path to the directory containing the configuration file.

CPU

The architecture of the Java virtual machine: x86 or x86_64

host

The name of the host machine, as returned by java.net.InetAddress.getLocalHost().getHostName()

os

The name of the host operating system: linux-gnu, win32, win64, or darwin

Environment variables

This section describes how to reference an environment variable in a DataSource configuration file.

Environment variables are referenced using the syntax below:

${ENV: environment variable }

The name of the environment variable must be preceded by the text ENV: and enclosed in curly braces, and the braces must be preceded by the $ character.

If the environment variable has not been defined, the syntax returns an empty string and a warning is logged to the console and log file.

The following example sets the runtime-user configuration item to the value of the USER environment variable:

runtime-user ${ENV:USER}

Expressions

This section describes how to use evaluated expressions in a configuration file. Expressions enable you to set a configuration item to a calculated value.

The expression macro uses the syntax below:

${EVAL: literal ${ config variable } ${ENV: env variable } @ config item + - * / literal ${ config variable } ${ENV: env variable } @ config item }

Each value in the expression can be a literal, a configuration variable, an environment variable, or a single-parameter configuration item.

All operators work with numeric operands, but the addition operator (+) is the only operator that works with string operands. If the subtraction (-), division (/), and multiplication (*) operators are used with a string operand, the DataSource raises an error.

When the addition operator (+) is used with string operands it concatenates the strings. If the addition operator is used with a numeric operand and a string operand, then the numeric operand is implicitly converted to a string.

Operators observe the rules of precedence described in the table below. You can use parentheses to group parts of an expression (not shown in the syntax diagram above).

Supported operators (in order of precedence)
Operators Description

( )

Parentheses (grouping)

* /

Multiplication and division

+ -

Addition and subtraction

Examples

The example below sets http-port to the sum of the PORT_BASE variable and 80 (=20080):

define PORT_BASE 20000
http-port ${EVAL:${PORT_BASE} + 80}

The example below sets https-port to the sum of the http-port configuration item and 1 (=8081):

http-port 8080
https-port ${EVAL:@http-port + 1}

Conditional flow control

DataSource libraries support conditional flow control constructions that have if/else/elseif/endif statements in the DataSource configuration.

The if statement uses the syntax below:

if conditional
  ...
elseif conditional (1)
  ...
else (2)
  ...
endif
1 Optional (0..n)
2 Optional (0..1)

Each conditional in an if statement can have a simple or complex form.

Simple conditionals

A simple conditional runs an application-defined test against a value.

test name literal ${ configuration variable } ${ENV: environment variable } ${EVAL: expression }

The value to be tested can be a literal, a configuration variable, an environment variable, or an expression.

DataSource applications support three tests by default:

C DataSource API simple conditional tests
Test Description

application

Returns TRUE if the test-value is equal to the value of the application-name configuration item.

hostname

Returns TRUE if the test-value is equal to the hostname.

os

Returns TRUE if the test-value is equal to the name of the operating system the application was compiled for. The test supports the following operating system names: linux-EL6-gnu, linux-EL7-gnu, win32, win64, darwin.

Java DataSource API simple conditional tests
Test Description

application

Returns TRUE if the test-value is equal to the value of the application-name configuration item.

host

Returns TRUE if the test-value is equal to the hostname as returned by the method java.net.InetAddress.getLocalHost().getHostName()

os

Returns TRUE if the test-value is equal to the name of the host operating system. The test supports the following operating system names: linux-gnu, win32, win64, darwin.

Additional simple conditional tests can be added to a DataSource application using the ds_config_set_test() function in the C DataSource API’s Configuration API. Currently no Caplin DataSource applications do so.

Example simple conditional

In the example below, the C DataSource configuration item jvm-location is set to a Windows path on a Windows host and to a Linux path on a Linux host:

Example of a simple conditional statement for a C DataSource application
if os win64
  # Microsoft Windows with Oracle Java 1.8.0
  jvm-location "C:\Program Files\Java\jre1.8.0_131\bin\server\jvm.dll"
elseif os linux-EL6-gnu
  # Red Hat Enterprise Linux 6 with Oracle Java 1.8.0
  jvm-location "/usr/java/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so"
endif

The Caplin Deployment Framework automatically sets a JVM_LOCATION variable based on the JAVA_HOME environment variable or a set of default values if the JAVA_HOME environment variable is not set. This allows you to set jvm_location with just one line of configuration:

jvm-location ${JVM_LOCATION}

Complex conditionals

A complex conditional consists of a single value or an expression that evaluates to a boolean value.

The syntax for a complex conditional is illustrated below:

literal ${ config variable } ${ENV: env variable } @ config item * / + - < <= > >= == != && and || or literal ${ config variable } ${ENV: env variable } @ config item

When writing a complex conditional with more than two operators, you can use parenthesis to control operator precedence (not shown in the syntax diagram above).

The following operators can be used inside a complex conditional:

Supported operators (in order of precedence)
Operators Description

( )

Parenthesis (grouping)

* /

Multiplication and division

+ -

Addition and subtraction

< <= > >=

Comparisons: less than, less than or equal to, more than, more than or equal to.

== !=

Comparisons: equal to, not equal to

&& and

Logical AND (&& or and can be used).

|| or

Logical OR (|| or or can be used).

If a complex conditional does not evaluate to a boolean value, the result is automatically converted to a boolean according to the rules below:

Automatic type conversion to boolean
Type Value Examples Effective boolean value

Numeric

> 0

10, "10"

TRUE

Numeric

< 0

-10, "-10"

TRUE

Numeric

0

0, "0"

FALSE

String

Empty

""

FALSE

String

Not empty

"example.com"

FALSE

Example complex conditionals

In the example below, https-port is set to 443 if the boolean configuration item https-enable has the value true:

if @https-enable
  https-port 443
endif

In the example below, https-port is set to 443 if the configuration item http-port is set to the standard HTTP port of 80:

if @http-port == 80
  https-port 443
endif

In the example below, https-enable is set to true and https-port is set to 443 if the configuration variable HTTPS has the value 1 and the configuration variable HOST_NAME has the value host1 or host2:

if ${HTTPS} == 1 and (${HOST_NAME} == host1 or ${HOST_NAME} == host2)
  https-enable
  https-port 443
endif

Including multiple configuration files

The include-file directive specifies that another configuration file has configuration settings for the DataSource application. The * wild card can be used in the relative path of the included file to include multiple configuration files.

The example include-file directive below includes all files with a .conf extension in the FX directory:

include-file FX/*.conf

Obtaining configuration from a webserver

The configuration for a DataSource application can be obtained from a web server by specifying the URL of the required configuration file in an include-file directive.

In the example below, a configuration file is downloaded from URL http://configurationserver/rttpd.conf:

include-file http://configurationserver/rttpd.conf

Only the HTTP protocol is supported.

By default, the file is retrieved using the wget command, which must be installed on the DataSource host. To use a different command to wget, see Changing the download command below.

Changing the download command

The default command used to download configuration files from webservers is wget -nv -O -. To use a different download command, specify a new value for the http-download-command configuration directive.

You can use any download utility that accepts the URL to download as its first parameter and writes the downloaded resource to STDOUT.

When specifying the name of the command to http-download-command, include any options required to suppress other output to STDOUT, such as progress indicators, error messages, and status codes.

For example, to set curl as the download utility, use the configuration below:

http-download-command "curl -s"

Note that curl’s silent option (-s) has been specified, which stops curl from writing progress and error messages to STDOUT.