Charting: cache and validate intraday data

This page provides information on the configuration items for caching and validating intraday data.

Contents:

Caching data

max-age

Maximum amount of time (in days) to keep caching a user-requested symbol after it was last discarded.

Default value:     10

cache-directory

Directory to be used to store the cached charting data files. '0' (zero) is representative of the root directory of the Transformer core application. 

Default value:     [current directory]

Example:

cache-directory       0/or/charts

add-fieldset

Defines how a Transformer handles a set of instruments and which fields should be cached for that type. See here for examples showing add-fieldset in use.

There are two ways of matching a fieldset to an instrument: using a set of regular expressions that match the subject of the instrument (using include-pattern and exclude-pattern), or using the contents of the "RECORDTYPE" field.

The RECORDTYPE field is a field available on RICs (Reuters Instrument Code) requested from Reuters that identifies the asset class and type of instrument. See here for the available values of RECORDTYPE.

The add-fieldset configuration item takes the following options:

add-fieldset
    exclude-pattern  [value]
    include-pattern  [value]
    recordtype       [value]
    add-field        [field] [size] [flags] [source-field] [group-code]
    cache-field      [value]
    rules            [value]
    pipeline-file    [value]
    pipeline-func    [value]
end-fieldset
Option Type Default Description
recordtype int -- The code identifying the record type. See here for a list of codes.
add-field string array --

A calculated field that is based on the value of a field in the update (source-field, below).

You can define multiple add-field options per add-fieldset stanza.

This option takes 5 parameters:

  • field — The name of the field
  • size — The size of the field in characters (used in calculating the required disk storage). If a field is bigger than this it will be truncated.
  • flags — Flags to specify which type of caching should be used. For a list of valid flags, see Charting: add-fieldset options.
  • source-field — The name of the field that is the source of the data.
  • group-code — The name of the group in which this field is included.
cache-field string array  -- Space-separated list of fields holding details such as the instrument's full name and permissioning information.
exclude-pattern string -- Regex (first match)
include-pattern string -- Regex (first match)
rules  string array --

A Reverse Polish notation (RPN) expression to validate an incoming price update before the fieldset caches it. If the RPN expression returns a non-zero value, the price update is not cached.

You can define multiple rules options per add-fieldset stanza.

You can also write validation rules using Lua and JavaScript. See the pipeline-file and pipeline-func options below.

For more information on writing RPN expressions for rules, see Charting: add-fieldset option values.

For example validation rules in RPN and Lua, see Charting: add-fieldset examples.

Important: do not include both rules and pipeline-* options in the same add-fieldset stanza.

pipeline-file string --

Name of a Transformer pipeline script file (Lua or JavaScript) used to validate an incoming price update before the fieldset caches it. Requires the pipeline-func option.

For example validation rules in RPN and Lua, see Charting: add-fieldset examples.

Important: do not include both rules and pipeline-* options in the same add-fieldset stanza.

pipeline-func string --

Name a Transformer pipeline function (Lua or JavaScript), used to validate an incoming price update before the fieldset caches it. Requires the pipeline-file option.

For example validation rules in RPN and Lua, see Charting: add-fieldset examples.

Important: do not include both rules and pipeline-* options in the same add-fieldset stanza.

Defining the instruments to cache

The add-fieldset stanza defines the fields to cache for an instrument or set of instruments.

To define the instruments that an add-fieldset stanza applies to, use the following three configuration options:

  • recordtype: the instrument's record-type
  • include-pattern: the instrument's RTTP subject
  • exclude-pattern: the RTTP subject of the instrument

Validating instrument values

You can refine the applicability of an add-fieldset stanza by specifying validation rules for specific fields in an update. If the validation rules are not met, then the fields defined in the add-fieldset stanza are not cached. You could, for example, reject updates in which the value of a specific field is zero or outside a percentage difference from the previously cached value.

To define validation rules for an add-fieldset stanza, use one of two sets of configuration options below:

  • rules: a Reverse Polish notation (RPN) expression
  • pipeline-file and pipeline-func: a named Transformer pipeline script and function name

Configuring Periods

To define periods, use the add-interval directive.

The options for the add-interval configuration item are:

Option Type Default Description
name string array - -
cache-size int - -
cache-interval int - Interval, in seconds, between data-points. Specify an interval of 0 to cache every change in value.
historic-routing [from subject] [to subject] - -

Examples

In example below, all requests for 1m (one minute) will be satisfied by the intraday cache within the Charting blade. The cache will contain 1000 data points and will be updated every 60 seconds.

add-interval
    name                 1m
    cache-interval       60
    cache-size           1000
    historic-routing     %1 file
end-interval

In the example below, any requests to chart an instrument under the /I directory for 1d (one day) will be forwarded onto a datasource request. So for example, when requesting /I/VOD.L with a 1d period from the charting blade, a request will be made for /TS1/dVOD.Ld to obtain the charting data.

add-interval
   name                  1d
   historic-routing      /I/%1  /TS1/d
end-interval