DataSource Prometheus metrics

This page lists the Prometheus metrics published by DataSource applications.

Prometheus metrics are not published by default. To enable this feature, see Publishing DataSource metrics in Prometheus format.

DataSource for C (DSDK) metrics

This section describes Prometheus metrics published by all C DataSource applications, including Liberator, Transformer, and the TREP Adapter, when prometheus-port is set.

This feature requires DataSource for C 7.1.16+ (Liberator 7.1.15+, Transformer 7.1.10+, TREP Adapter 7.1.3+).

The Prometheus metrics endpoint URI is /metrics.

For descriptions of additional metrics published by specific C DataSource applications, see the links below:

Process metrics

C DataSource applications publish the following metrics on the operating system process that runs the DataSource.

These metrics are only available on RHEL builds of C DataSources.

process_cpu_seconds_total

[gauge] Total user and system CPU time spent in seconds.

process_max_fds

[gauge] Maximum number of open file descriptors.

process_open_fds

[gauge] Number of open file descriptors.

process_start_time_seconds

[gauge] Start time of the process since the unix epoch in seconds.

process_virtual_memory_bytes

[gauge] Virtual memory size in bytes.

process_virtual_memory_max_bytes

[gauge] Maximum amount of virtual memory available in bytes.

System metrics

C DataSources publish the following CPU, memory, and process metrics, where appname is the name of the DataSource application, as defined by the application-name configuration item.

appname_system_cputime_total

[gauge] The total amount of time (seconds) the CPU has spent running the DataSource process. This value is the sum of appname_system_system_cputime_total and appname_system_user_cputime_total.

appname_system_system_cputime_total

[gauge] The total amount of time (seconds) the CPU has spent running kernel-space code for the DataSource process.

appname_system_user_cputime_total

[gauge] The total amount of time (seconds) the CPU has spent running user-space code for the DataSource process.

appname_system_process_uptime

[gauge] Elapsed 'wall clock' time (seconds) since the DataSource process was started.

appname_system_memory_usage

[gauge] The total amount of virtual memory (kilobytes) used by the DataSource process, as reported by the VmRSS value in /proc/<pid>/status.

appname_system_cpu_usage

[gauge] Average CPU percentage used by the DataSource process in the period configured by the configuration option process-usage-period (default 10 seconds).

DataSource peer metrics

Metrics for DataSource peer connections, where appname is the name of the DataSource application, as defined by the application-name configuration item.

appname_peer_global_bytes_read

[gauge] Total number of bytes read from all peer connections.

appname_peer_global_bytes_read_rate

[gauge] The rate data is received (bytes/second) from all peer connections, measured over the sample period defined by peer-monitor-interval.

appname_peer_global_bytes_written

[gauge] Total number of bytes written to peer connections.

appname_peer_global_bytes_written_rate

[gauge] The rate data is written (bytes/second) to all peer connections, measured over the sample period defined by peer-monitor-interval.

appname_peer_global_messages_read

[gauge] Total number of messages read from peer connections.

appname_peer_global_messages_read_rate

[gauge] The rate messages are received (messages/second) from all peer connections, measured over the sample period defined by peer-monitor-interval.

appname_peer_global_messages_written

[gauge] Total number of messages written to peer connections.

appname_peer_global_messages_written_rate

[gauge] The rate messages are written (messages/second) to all peer connections, measured over the sample period defined by peer-monitor-interval.

Data service metrics

C DataSources publish the following metrics for each data service, where appname is the name of the DataSource application as defined by application-name and servicename is the name of the data service.

appname_service_servicename_average_response_time

[gauge] The average time (milliseconds) taken for the data service to respond to a subscription request, measured over the sample period defined by peer-monitor-interval.

appname_service_servicename_bytes_read

[gauge] Total number of bytes read from the data service.

appname_service_servicename_bytes_read_rate

[gauge] The rate at which data is received (bytes/second) from the data service, measured over the sample period defined by peer-monitor-interval.

appname_service_servicename_bytes_written

[gauge] Total number of bytes written to the data service.

appname_service_servicename_bytes_written_rate

[gauge] The rate at which data is written (bytes/second) to the data service, measured over the sample period defined by peer-monitor-interval.

appname_service_servicename_messages_read

[gauge] The total number of messages received from the data service.

appname_service_servicename_messages_read_rate

[gauge] The rate at which messages are received (messages/second) from the data service, measured over the sample period defined by peer-monitor-interval.

appname_service_servicename_messages_written

[gauge] The total number of messages written to the data service.

appname_service_servicename_messages_written_rate

[gauge] The rate at which messages are written (messages/second) to the data service, measured over the sample period defined by peer-monitor-interval.

appname_service_servicename_response_time

[gauge] The time taken (seconds) for the data service to respond to the first subscription request. This value only changes if the object is discarded and requested again.

appname_service_servicename_state

[gauge] The state of the data service. It has two possible values: 1 for up, 0 for down.

DataSource for Java metrics

Prometheus metrics published by all Java DataSources (7.1.14+) when prometheus-port is set.

The Prometheus metrics endpoint URI is /metrics. This can be customised for Java DataSources; see prometheus-endpoint.

Java DataSource: JVM metrics

Metrics for the Java virtual machine (JVM) running the DataSource.

jvm_buffer_count_buffers

[gauge] An estimate of the number of buffers in the pool.

jvm_buffer_memory_used_bytes

[gauge] An estimate of the memory that the Java virtual machine is using for this buffer pool

Tags for jvm_buffer_memory_used_bytes:

  • id="direct"

  • id="mapped"

jvm_buffer_total_capacity_bytes

[gauge] An estimate of the total capacity of the buffers in this pool

jvm_gc_live_data_size_bytes

[gauge] Size of old generation memory pool after a full GC

jvm_gc_max_data_size_bytes

[gauge] Max size of old generation memory pool

jvm_gc_memory_allocated_bytes_total

[counter] Incremented for an increase in the size of the young generation memory pool after one GC to before the next

jvm_gc_memory_promoted_bytes_total

[counter] Count of positive increases in the size of the old generation memory pool before GC to after GC

jvm_gc_pause_seconds

[summary] Time spent in GC pause

Tags for jvm_gc_pause_seconds_count:

  • action="end of major GC", cause="Metadata GC Threshold"

  • action="end of minor GC", cause="Metadata GC Threshold"

  • action="end of minor GC", cause="Allocation Failure"

Tags for jvm_gc_pause_seconds_sum:

  • action="end of major GC", cause="Metadata GC Threshold"

  • action="end of minor GC", cause="Metadata GC Threshold"

  • action="end of minor GC", cause="Allocation Failure"

jvm_gc_pause_seconds_max

[gauge] Time spent in GC pause

Tags for jvm_gc_pause_seconds_max:

  • action="end of major GC", cause="Metadata GC Threshold"

  • action="end of minor GC", cause="Metadata GC Threshold"

  • action="end of minor GC", cause="Allocation Failure"

jvm_memory_committed_bytes

[gauge] The amount of memory in bytes that is committed for the Java virtual machine to use

Tags for jvm_memory_committed_bytes:

  • area="heap", id="PS Survivor Space"

  • area="heap", id="PS Old Gen"

  • area="heap", id="PS Eden Space

  • area="nonheap", id="Metaspace"

  • area="nonheap", id="Code Cache"

  • area="nonheap", id="Compressed Class Space"

jvm_memory_max_bytes

[gauge] The maximum amount of memory in bytes that can be used for memory management

Tags for jvm_memory_max_bytes:

  • area="heap", id="PS Survivor Space"

  • area="heap", id="PS Old Gen"

  • area="heap", id="PS Eden Space

  • area="nonheap", id="Metaspace"

  • area="nonheap", id="Code Cache"

  • area="nonheap", id="Compressed Class Space"

jvm_memory_used_bytes

[gauge] The amount of used memory

jvm_threads_daemon_threads

[gauge] The current number of live daemon threads

jvm_threads_live_threads

[gauge] The current number of live threads including both daemon and non-daemon threads

jvm_threads_peak_threads

[gauge] The peak live thread count since the Java virtual machine started or peak was reset

jvm_threads_states_threads

[gauge] The current number of threads having NEW state

Tags for jvm_threads_states_threads:

  • state="runnable"

  • state="blocked"

  • state="waiting"

  • state="time-waiting"

  • state="new"

  • state="terminated"

process_cpu_usage

[gauge] The "recent cpu usage" for the Java Virtual Machine process

process_start_time_seconds

[gauge] Start time of the process since unix epoch.

process_uptime_seconds

[gauge] The uptime of the Java virtual machine

system_cpu_count

[gauge] The number of processors available to the Java virtual machine

system_cpu_usage

[gauge] The "recent cpu usage" for the whole system

system_load_average_1m

[gauge] The sum of the number of runnable entities queued to available processors and the number of runnable entities running on the available processors averaged over a period of time

Java DataSource: peer metrics

Metrics for DataSource peer connections, where appname is the name of the DataSource application, as defined by the application-name configuration item.

appname_discard.average

[gauge] Average duration of discarded requests from a DataSource peer. To configure the update frequency for this metric, see peer-debug-metrics-interval.

appname_discard.count

[counter] Total number of discarded requests from a DataSource peer. To configure the update frequency for this metric, see peer-debug-metrics-interval.

appname_discard.max

[gauge] Duration in milliseconds of the longest discarded request from a DataSource peer. To configure the update frequency for this metric, see peer-debug-metrics-interval.

appname_executor.queue.length

[counter] Total number of requests from a DataSource peer that are pending execution. To configure the update frequency for this metric, see peer-debug-metrics-interval.

appname_netty.bytes.before.writable

[gauge] [Netty] How many bytes remain in underlying buffers until a thread can begin to process queued requests. See Channel.bytesBeforeWritable() for more information. To configure the update frequency for this metric, see peer-debug-metrics-interval.

appname_netty.bytes.before.unwritable

[gauge] [Netty] How many bytes can be written to underlying buffers until a thread must add new requests to the queue rather than process them immediately. See Channel.bytesBeforeUnwritable() for more information. To configure the update frequency for this metric, see peer-debug-metrics-interval.

appname_peer_global_bytes_read

[gauge] Total number of bytes read from all peer connections.

appname_peer_global_bytes_read_rate

[gauge] The rate data is received (bytes/second) from all peer connections (sample period: 1 second).

appname_peer_global_bytes_written

[gauge] Total number of bytes written to peer connections.

appname_peer_global_bytes_written_rate

[gauge] The rate data is written (bytes/second) to all peer connections (sample period: 1 second).

appname_peer_global_messages_read

[gauge] Total number of messages read from peer connections.

appname_peer_global_messages_read_rate

[gauge] The rate messages are received (messages/second) from all peer connections (sample period: 1 second).

appname_peer_global_messages_written

[gauge] Total number of messages written to peer connections.

appname_peer_global_messages_written_rate

[gauge] The rate messages are written (messages/second) to all peer connections (sample period: 1 second).

appname_request.average

[gauge] Average duration of requests from a DataSource peer. To configure the update frequency for this metric, see peer-debug-metrics-interval.

appname_request.count

[counter] Total number of requests made from a DataSource peer. To configure the update frequency for this metric, see peer-debug-metrics-interval.

appname_request.max

[gauge] Duration in milliseconds of the longest request from a DataSource peer. To configure the update frequency for this metric, see peer-debug-metrics-interval.

See also: