Configure DataSource Prometheus monitoring
This page describes how to publish metrics in Prometheus format.
Publishing DataSource metrics in Prometheus format
Version requirements:
-
DataSource for C 7.1.16+ (Liberator 7.1.15+, Transformer 7.1.10+, TREP Adapter 7.1.3+)
-
DataSource for Java 7.1.14+
DataSource metrics are disabled by default. To publish DataSource metrics, follow the steps below:
-
Set
prometheus-port
to a free port on the DataSource’s host:prometheus-port 48557
-
Restart the DataSource
-
Open a web browser and navigate to http://<datasource_host>:<prometheus_port>/metrics. You will see output similar to the excerpt below:
Example Liberator DataSource metrics in Prometheus format# HELP rttpd_user_global_login_fail counter # TYPE rttpd_user_global_login_fail counter rttpd_user_global_login_fail 0 # HELP rttpd_session_global_discards gauge # TYPE rttpd_session_global_discards gauge rttpd_session_global_discards 0
DataSource metric references:
Publishing JVM metrics in Prometheus format
DataSource metrics do not include metrics on the JVM associated with a DataSource. To publish JVM metrics, we recommend that you configure the DataSource’s JVM to load the Prometheus project’s JMX Exporter as a JVM Java agent.
Configuring a C DataSource to run the Prometheus JMX Exporter
This section describes how to configure a C DataSource’s embedded JVM to load the Prometheus JMX Exporter as a JVM Java agent.
The instructions in this section use Liberator as an example and assume that you have already deployed or activated a blade that enables Liberator’s embedded JVM. For example, the Liberator’s LiberatorJMX blade or Caplin’s PermissioningService module.
Follow the steps below:
-
Download the latest Prometheus JMX Exporter Java Agent Jar file to your Deployment Framework’s
tools
directory. -
In
global_config/environment.conf
, define a convenience variable for the location of the JMX Exporter Jar file:File: global_config/environment.confdefine JMX_EXPORTER_JAR
/tools/jmx_prometheus_javaagent-0.17.2.jarThe latest version of JMX Exporter at the time of writing is 0.17.2.
-
Add the following line of configuration to Liberator’s
java.conf
override file:File: global_config/overrides/servers/Liberator/etc/java.confjvm-options -javaagent:
=jmx_exporter_port: /jmx-exporter.yamlwhere jmx_exporter_port is a free port on the host. If you are running multiple DataSources on the host, then assign each JMX Exporter a unique port.
-
Create a configuration file for the JMX Exporter in the overrides directory for Liberator:
File: global_config/overrides/servers/Liberator/etc/jmx-exporter.yaml--- rules: - pattern: ".*"
-
(Re)start Liberator
$ ./dfw start
-
Open a web browser and navigate to http://liberator_host:jmx_exporter_port. If the Java agent is running correctly, then you will see output similar to the (truncated) output below:
# HELP jvm_threads_current Current thread count of a JVM # TYPE jvm_threads_current gauge jvm_threads_current 22.0 # HELP jvm_threads_daemon Daemon thread count of a JVM # TYPE jvm_threads_daemon gauge jvm_threads_daemon 19.0 # HELP jvm_threads_peak Peak thread count of a JVM # TYPE jvm_threads_peak gauge jvm_threads_peak 28.0
Configuring a Java DataSource to run the Prometheus JMX Exporter
This section describes how to configure a Java DataSource’s JVM to load the Prometheus JMX Exporter as a JVM Java agent.
Follow the steps below:
-
Download the latest Prometheus JMX Exporter Java Agent jar file to your Deployment Framework’s
tools
directory. -
Make the following modifications (highlighted) to the Java DataSource’s startup script:
kits/bladename/bladename-version/DataSource/bin/start-jar.sh
. The example script below is adapted from Caplin’s Pricing Adapter Template (see start-jar.sh):#!/bin/bash BLADENAME=MyPricingAdapter # Prometheus JMX Exporter jar location, port, and config file location PROM_AGENT=${CONFIG_BASE}../tools/jmx_prometheus_javaagent-0.17.2.jar (1) PROM_AGENT_PORT=5580 (2) PROM_AGENT_CONFIG=${CONFIG_BASE}overrides/${BLADENAME}/etc/jmx-exporter.yaml if [ "$1" = "CONFREADER" ]; then shift confreading=1 jar=`ls "$BINARY_ROOT"/lib/datasource*.jar|head -1` else confreading=0 jar=`ls "$BINARY_ROOT"/lib/$BLADENAME*.jar|head -1` classpath="${BINARY_ROOT}/lib/*" echo "Classpath: $jar" fi if [ $confreading = 1 ]; then java -jar "$jar" "$@" exit $? else java \ -javaagent:${PROM_AGENT}=${PROM_AGENT_PORT}:${PROM_AGENT_CONFIG} \ -cp "$classpath" \ -jar "$jar" "$@" 2> "$LOGDIR"/java-$BLADENAME.log >/dev/null & echo $! fi
1 The latest version of JMX Exporter at the time of writing is 0.17.2 2 Set PROM_AGENT_PORT
to a free port on the host. Each JMX Exporter running on the host requires its own unique port. -
Create a new configuration file for the JMX Exporter in the overrides directory for the adapter:
File: global_config/overrides/<bladename>/etc/jmx-exporter.yaml--- rules: - pattern: ".*"
-
(Re)start the Java DataSource:
$ ./dfw start
-
Open a web browser and navigate to http://<liberator-host>:5580/. If the Java agent is running correctly, then you will see output similar to the (truncated) output below:
# HELP jvm_gc_collection_seconds Time spent in a given JVM garbage collector in seconds. # TYPE jvm_gc_collection_seconds summary jvm_gc_collection_seconds_count{gc="PS Scavenge",} 1.0 jvm_gc_collection_seconds_sum{gc="PS Scavenge",} 0.005 jvm_gc_collection_seconds_count{gc="PS MarkSweep",} 1.0 jvm_gc_collection_seconds_sum{gc="PS MarkSweep",} 0.013 # HELP jvm_classes_loaded The number of classes that are currently loaded in the JVM # TYPE jvm_classes_loaded gauge jvm_classes_loaded 2533.0