Charting: cache and validate intraday data

This page provides information on the configuration items for caching and validating intraday data.

Contents:

Caching data

max-age

Maximum amount of time (in days) to keep caching a user-requested symbol after it was last discarded.

Default value:     10

cache-directory

Directory to be used to store the cached charting data files. '0' (zero) is representative of the root directory of the Transformer core application. 

Default value:     [current directory]

Example:

cache-directory       0/or/charts

add-fieldset

Defines how a Transformer handles a set of instruments and which fields should be cached for that type. See here for examples showing add-fieldset in use.

There are two ways of matching a fieldset to an instrument: using a set of regular expressions that match the subject of the instrument (using include-pattern and exclude-pattern), or using the contents of the "RECORDTYPE" field.

The RECORDTYPE field is a field available on RICs (Reuters Instrument Code) requested from Reuters that identifies the asset class and type of instrument. See here for the available values of RECORDTYPE.

The add-fieldset configuration item takes the following options:

add-fieldset
    exclude-pattern  [value]
    include-pattern  [value]
    recordtype       [value]
    add-field        [field] [size] [flags] [source-field] [group-code]
    cache-field      [value]
    rules            [value]
    pipeline-file    [value]
    pipeline-func    [value]
end-fieldset
Option Type Default Description
recordtype int [no default] The code identifying the record type. See here for a list of codes.
add-field string array [no default]

A calculated field that is based on the value of a field in the update (source-field, below).

You can define multiple add-field options per add-fieldset stanza.

This option takes 5 parameters:

  • field — The name of the field
  • size — The size of the field in characters (used in calculating the required disk storage). If a field is bigger than this it will be truncated.
  • flags — Flags to specify which type of caching should be used. For a list of valid flags, see Charting: add-fieldset options.
  • source-field — The name of the field that is the source of the data.
  • group-code — The name of the group in which this field is included.
cache-field string array  [no default] Space-separated list of fields holding details such as the instrument's full name and permissioning information.
exclude-pattern -- -- Regex (first match)
include-pattern -- -- Regex (first match)
rules  string array [no default]

A Reverse Polish notation (RPN) expression. If the expression returns a non-zero value, the fieldset is not cached.

You can define multiple rules options per add-fieldset stanza.

For more information on writing RPN expressions for rules, see Charting: add-fieldset option values. For a comparison of writing validation rules using RPN and Lua, see Charting: add-fieldset examples.

Important: do not include both rules and pipeline-* options in the same add-fieldset stanza.

pipeline-file string --

Name of a script file containing a pipeline function (defined by pipeline-func) used to validate incoming values.

For a comparison of writing validation rules using RPN and Lua, see Charting: add-fieldset examples.

Important: do not include both rules and pipeline-* options in the same add-fieldset stanza.

pipeline-func string --

Pipeline function, in the file defined by pipeline-file, used to validate incoming values.

For a comparison of writing validation rules using RPN and Lua, see Charting: add-fieldset examples.

Important: do not include both rules and pipeline-* options in the same add-fieldset stanza.

Validating data

The add-fieldset stanza defines the fields to cache for an instrument or set of instruments.

To define the instruments that an add-fieldset stanza applies to, use the following three configuration options:

  • recordtype: the instrument's record-type
  • include-pattern: the instrument's RTTP subject
  • exclude-pattern: the RTTP subject of the instrument

You can further refine the applicability of an add-fieldset stanza by specifying validation rules for specific fields in an update. If the validation rules are not met, then the fields defined in the add-fieldset stanza are not cached. You could, for example, reject updates in which the value of a specific field is zero or outside a percentage difference from the previously cached value.

To define validation rules for an add-fieldset stanza, use one of two sets of configuration options below:

  • rules: a Reverse Polish notation (RPN) expression
  • pipeline-file and pipeline-func: a named Transformer pipeline script and function name

Configuring Periods

To define periods, use the add-interval directive.

The options for the add-interval configuration item are:

Option Type Default Description
name string array - -
cache-size int - -
cache-interval int - Interval, in seconds, between data-points. Specify an interval of 0 to cache every change in value.
historic-routing [from subject] [to subject] - -

Examples

In example below, all requests for 1m (one minute) will be satisfied by the intraday cache within the Charting blade. The cache will contain 1000 data points and will be updated every 60 seconds.

add-interval
    name                 1m
    cache-interval       60
    cache-size           1000
    historic-routing     %1 file
end-interval

In the example below, any requests to chart an instrument under the /I directory for 1d (one day) will be forwarded onto a datasource request. So for example, when requesting /I/VOD.L with a 1d period from the charting blade, a request will be made for /TS1/dVOD.Ld to obtain the charting data.

add-interval
   name                  1d
   historic-routing      /I/%1  /TS1/d
end-interval