Refiner subject syntax
This page describes the syntax of subjects that are processed by Refiner.
If you are system administrator, this page will help you decipher Refiner subjects in log files.
If you are a software developer and you’re using StreamLink, see Access Refiner through StreamLink.
If you are a software developer and you’re using Caplin Trader’s Grid module, see Configure sorting and filtering.
Refiner limitations
Avoid filtering or sorting on record fields that are subject to frequent updates. Frequent updates can have an adverse affect on the performance of Caplin Refiner, Liberator, and the requesting clients.
Refiner subject format
To use Refiner to filter, sort, and group data in a container, prepend the container’s subject with /FILTER
and append the container’s subject with ?
followed by a string of Refiner parameters.
Syntax diagram:
At least one of the following parameters is required: filter
, sort
, groupby
The mapsuffix
parameter is normally added dynamically, when it is required, by a permissioning adapter. See The 'mapsuffix' parameter.
Multiple instances of the same parameter are not allowed, with the exception of the sort
parameter, which from Refiner 7 may be specified multiple times to construct a multi-column sort. See The 'sort' parameter.
An example Refiner subject is shown below. It requests the /CTR/FI/ALL container, sorted and grouped by the Description field.
/FILTER/CTR/FI/ALL?sort=Description:text:ascending;groupby=Description
The 'filter' parameter
The filter parameter comprises one or more field-matching criteria.
A single field-matching criterion has the following syntax:
Meaning | Text fields | Numeric fields | Notes | |
---|---|---|---|---|
|
equal to |
Defaults to text comparison. For a numeric comparison, prefix the field name with |
||
|
not equal to |
Defaults to text comparison. For a numeric comparison, prefix the field name with |
||
|
equal to (numeric) |
|||
|
not equal to (numeric) |
|||
|
less than |
Refiner 7.1.0+ |
StreamLink’s |
|
|
greater than |
Refiner 7.1.0+ |
StreamLink’s |
|
|
less than or equal to |
Refiner 7.1.0+ |
StreamLink’s |
|
|
more than or equal to |
Refiner 7.1.0+ |
StreamLink’s |
|
|
regular expression match |
Usually used for text fields |
||
|
regular expression match |
Usually used for text fields |
You can join multiple field-matching criteria with logical AND ('&') and logical OR ('|'). There’s no implicit precedence between the logical operators, so you need to establish precedence using parenthesis ('(' and ')').
CpnRate>6
CpnRate>6&CpnRate<10
MaturityDate=20130328|(CpnRate>6&CpnRate<10)
If the field name or comparison value in a filter criterion includes a Refiner comparison operator (see table above), a Refiner logical operator ('&' or '|'), or the Refiner parameter separator (';'), then these characters must be encoded using a Refiner escape sequence. See Special characters.
Special characters
The following characters have a special meaning in Refiner filter expressions. To include these characters as literals in filter expressions (in a field name or in a value), use the following escape sequences:
Character | Escape sequence |
---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For example, to filter a ccy
field using the regular expression USD(CHF|CAD)
, use the filter parameter below (escape sequences highlighted):
filter=ccy#USD@1CHF@4CAD@2
StreamLink contains classes that make building the Refiner parameter string easier. See Access Refiner through StreamLink. |
The 'sort' parameter
The sort parameter comprises a field name, a field data type, and a sort order:
For example, to sort the 'MaturityDate' field in ascending numerical order, use the sort parameter below:
MaturityDate:number:ascending
From Refiner 7, you can specify more than one sort parameter in a Refiner parameter string. The sort parameters are applied in the order they are supplied, and create a multi-column sort:
/FILTER/CTR/FI/ALL?sort=MaturityDate:number:ascending;sort=CpnRate:number:descending
If two or more records have the same value in the sort field(s), the records are sorted into their natural order (the order in which the records' integration adapter would return the records unsorted to the client).
The 'groupby' parameter
The groupby parameter comprises a single value: the field name to group records by.
Grouping adds extra records to the container that act as headers for blocks of records with the same field value. For example, if you sort the /CTR/FI/All
container by the Description
field and group the records by the Description
field, the returned container looks like this:
/FILTER/META/CTR/FI/ALL/GROUPHEADER/groupby=Description/AUSTRIA SubHeaderText=AUSTRIA /FI/EU/A Description=AUSTRIA CpnRate=4.35 MaturityDate=20140715 BidPrice=103.948 ... /FI/EU/B Description=AUSTRIA CpnRate=6.25 MaturityDate=20270715 BidPrice=122.890 ... /FILTER/META/CTR/FI/ALL/GROUPHEADER/groupby=Description/GERMANY SubHeaderText=GERMANY Description=GERMANY /FI/EU/C CpnRate=4.92 MaturityDate=20130328 BidPrice=101.859 ... /FILTER/META/CTR/FI/ALL/GROUPHEADER/groupby=Description/US TREASURY SubHeaderText=US TREASURY /FI/US/A Description=US TREASURY CpnRate=11.25 MaturityDate=20141115 BidPrice=... /FI/US/B Description=US TREASURY CpnRate=11.75 MaturityDate=20150215 BidPrice=... /FI/US/C Description=US TREASURY CpnRate=10.62 MaturityDate=20150815 BidPrice=...
Each of the header records (/FILTER/META/CTR/FI/ALL/GROUPHEADER/
and so on) contains one field, SubHeaderText
, which contains the value of the field that you are grouping by. When the client application receives the sorted and grouped container, it typically displays the header rows in the grid with different styling to the data rows.
Within Streamlink, the exact way in which record grouping is added to a container subscription depends on which StreamLink library you are using. For details, consult the API Reference document for your particular StreamLink implementation. |
The 'mapsuffix' parameter
The 'mapsuffix' parameter is added to a Refiner subject dynamically by a permissioning integration adapter.
The parameter comprises a single value:
The suffix is appended to the subjects of the items in the container and is used to implement price tiers. For more information on 'mapsuffix', see Set Refiner user permissions and subject mappings.
The container placeholder
The sequence of subscribing to a container, subscribing to all the records in it, and then processing each of the records in Caplin Refiner can take longer than the active request timeout configured in Liberator (see source-request-timeout
, add-peer
: request-timeout
and add-data-service
: request-timeout
). If Refiner were to wait until it had processed all the results before sending a response, the active request timeout might trigger first and Liberator would then discard the container. To prevent this happening, as soon as Caplin Refiner receives a filter or sort request, it sends back a container response with a single special subject name in it: /FILTER/META/PLACEHOLDER
. This is called the container placeholder record. It tells the Liberator that Refiner has received the request, preventing the container subscription from being discarded.
Handling container placeholders in the client
Typically a client application displays a loading indicator while waiting for a response to a sort or filter request, and it removes the loading indicator when a response is received. If the first response received is a container of size 1
that contains a placeholder record, the client should ignore the response and leave the loading indicator in place. The next container update received will either be a container image of the results, or an explicit removal of the placeholder record that indicates there are no matching results. The client should respond by removing the loading indicator.
Caplin Refiner explicitly removes the placeholder record when the time taken to filter the container exceeds the timeout defined in the Refiner configuration property placeholder.timeout
. In this case, the client can still receive results from the filter request until such time as it discards the filter.
See also:
-
How can I… Implement custom sorting and filtering in Refiner
-
How can I… Deploy the refiner service module
-
How can I… Access Refiner through StreamLink
-
How can I… Configure Refiner
-
How can I… Configure sorting and filtering (in Caplin Trader 4 grids).
-
How can I… Set Refiner user permissions and subject mappings
-
Reference: Refiner configuration properties