Dynamic Peers

Dynamic Peers allows you to add new instances of adapters to a running deployment without downtime to add extra add-peer configuration to the peers they connect to.

Available from: Platform 8

Overview

In this overview of Dynamic Peers, we’ll look at three different ways of managing peer connections:

  • Statically defined peer connections

  • Dynamic incoming peer connections (Dynamic Peers)

  • Dynamic incoming and outgoing peer connections (Discovery)

Statically defined peer connections

In a deployment with statically defined peer connections, connecting peers require corresponding add-peer configuration items: one indicating the peer to connect to, and one indicating the peer to accept a connection from.

Liberator1192.168.1.100PricingAdapter1datasrc-local-label Liberator1 # Incoming peer connectionadd-peerremote-label PricingAdapter1end-peer add-data-serviceservice-name PricingServiceadd-source-groupadd-priorityremote-label PricingAdapter1end-priorityend-source-groupend-data-servicedatasrc-local-label PricingAdapter1 # Outgoing peer connectionadd-peeraddr 192.168.1.100port 14001end-peer
Statically defined peer connections

In the deployment above, scaling the number of instances of Liberator or PricingAdapter requires downtime to add extra add-peer configuration to the peers.

Dynamic incoming connections (Dynamic Peers)

Caplin Platform 8 introduces a new feature, Dynamic Peers, that provides a secure way to avoid the requirement for add-peer definitions of incoming peer connections. It does this with an extended DataSource peer handshake, in which the connecting peer lists the services it provides. If the accepting peer is a consumer of a service that the connecting peer provides, then the connection succeeds. For added security, the handshake can be secured with an optional API key.

Liberator1192.168.1.100PricingAdapter1datasrc-local-label Liberator1 add-data-serviceservice-namePricingServiceadd-source-groupadd-priorityremote-label-regexPricingAdapter[0-9]+end-priorityend-source-groupend-data-servicedatasrc-local-label PricingAdapter1 # Outgoing peer connectionadd-peeraddr 192.168.1.100port 14001end-peer discovery-provide-servicePricingService
Platform 8 Dynamic Peers

In the deployment above, the number of PricingAdapter instances can be scaled dynamically at runtime.

The number of Liberator instances cannot be scaled dynamically at runtime. Downtime is required to add extra add-peer configuration to PricingAdapter instances.

The names of services provided by a peer are defined by discovery-provide-service and the service-name option of add-provided-data-service (see Dynamic Services).

The names of services consumed by a peer are defined by discovery-require-service (for Dynamic Services provided by connecting peers) and the service-name option of statically defined add-data-service blocks.

Dynamic incoming and outgoing connections (Discovery)

Discovery removes the requirement for static add-peer configuration for co-located peers, and all co-located peers can be dynamically scaled at runtime:

Liberator1PricingAdapter1Discovery1192.168.1.200datasrc-local-label Liberator1 discovery-addr 192.168.1.200 add-data-serviceservice-namePricingServiceadd-source-groupadd-priorityremote-label-regexPricingAdapter[0-9]+end-priorityend-source-groupend-data-servicedatasrc-local-label PricingAdapter1 discovery-addr 192.168.1.200 discovery-provide-servicePricingService
Discovery deployment

When providers of services connect to Discovery they are given a dynamic list of peers interested in consuming their services, making it easy for providing peers to find and connect to their consumers. No add-peer configuration is required; peers only need the location of the deployment’s Discovery server.

The services provided by a peer are defined by discovery-provide-service.

The services consumed by a peer are defined by discovery-require-service and the service-name option of add-data-service blocks.

Migrating to Dynamic Peers

Follow the steps in Enabling Dynamic Peers in the DataSource documentation.

Choosing Dynamic Peers or Discovery’s peer discovery

Dynamic Peers was introduced in Platform 8 to complement, not replace, Discovery’s peer discovery. Thus, the choice is not always either/or. In some use cases, it is advantageous to use both.

When should I deploy Discovery?

  • If you need to dynamically scale Liberator and/or Transformer, then deploy Discovery.

  • If you currently use IP-bound licensing for Caplin Platform products, and you need to deploy to an environment in which IP addresses are not fixed (such as container-based deployment platforms), then deploy Discovery to take advantage of its centralised license management based on instance numbers.

  • If your deployment uses a Transformer cluster, then consider deploying Discovery, which configures Transformer clustering automatically for you.

  • If you want to take advantage of Discovery’s web interface that monitors bandwidth and latency across all connections in your deployment, then deploy Discovery.

When might Dynamic Peers meet all my scaling requirements?

  • Dynamic Peers only offers dynamic scaling of adapters, but in our experience it is adapters, rather than Liberator and Transformer, that most often benefit from dynamic scaling. Thus Dynamic Peers can bring the advantages of dynamic scaling to many deployments without the requirement to migrate to Discovery.

When do I need both Discovery and Dynamic Peers?

  • Discovery’s peer discovery only works for co-located peers. Peers located off site do not have access to Discovery and so would not ordinarily be dynamically scalable. Platform 8’s Dynamic Peers fills this feature gap.

    Data Centre 1Data Centre 2Liberator1192.168.1.100PricingAdapter1Discovery1192.168.1.200TradingAdapter1lib1.example.comdatasrc-local-label Liberator1 discovery-addr 192.168.1.200 add-data-serviceservice-namePricingServiceadd-source-groupadd-priorityremote-label-regexPricingAdapter[0-9]+end-priorityend-source-groupend-data-service add-data-serviceservice-nameTradingServiceadd-source-groupadd-priorityremote-label-regexTradingAdapter[0-9]+end-priorityend-source-groupend-data-servicedatasrc-local-label PricingAdapter1 discovery-addr 192.168.1.200 discovery-provide-servicePricingServicedatasrc-local-label TradingAdapter1 # Outgoing peer connectionadd-peeraddr lib1.example.comport 14001end-peer discovery-provide-serviceTradingService
    Multi-site deployment

    In the deployment above, Platform 8’s Dynamic Peers allows the number of TradingAdapter instances to be scalable at runtime.