View packet logs using logcat

The logs produced by most Caplin Platform components are simple text files that can be viewed using a suitable text display utility or text editor, such as the Linux commands cat, more, and vim. However, the packet logs are in binary format and must be viewed using the Caplin logcat utility. Here’s how to do this.

Requirements

If the logcat command’s -F option is used, and the fields file referenced by the -f option contains the CONFIG_BASE variable, then you must enable the Deployment Framework’s command-line completion extension. See Setting up dfw command completion.

From version 7.1 of the Caplin Platform, packet logs use a new binary format to accommodate high-resolution timestamps. Packet logs created by DataSource 7.1 components can only be read by logcat 7.1 or greater. For more information on high-resolution timestamps in Caplin Platform 7.1, see Timestamp resolution.

Running logcat

There’s a copy of the logcat utility in the bin directory of each installed Liberator, Transformer and Integration Adapter.

In the Caplin Deployment Framework, you’ll find logcat in the following places:

  • <Framework-root>/servers/Liberator/bin

  • <Framework-root>/servers/Transformer/bin

  • <Framework-root>/kits/<Adapter-blade-name>/<Adapter-blade-name>/DataSource/bin

where <Framework-root> is the Deployment Framework’s topmost directory.

You can generally use any instance of logcat in your Framework to view any of the packet logs produced by any component running under the Framework. However, we recommend that you normally use the latest logcat whenever possible; that is, the logcat provided in the latest kit version out of all the component kits you’ve deployed in the Framework.

You use logcat in a similar way to the standard Linux cat command. So, if you’re in the directory where logcat resides, you enter a command of the form:

./logcat [options] <path-and-name-of-binary-log-file>

The logcat utility takes the following options:

Option Usage

-h

--help

Displays detailed information about the available logcat options.

-F

--print-field-names

Displays the names of the fields in each log record (the default is to just print the field numbers). The field names are obtained from the component’s fields file (see -f option).

If the fields file, or a file referenced by the fields file, uses the environment variable CONFIG_BASE, then you must enable the Deployment Framework’s command-line completion extension. See Setting up dfw command completion.

-f <fields-file>

--fields-file <fields-file>

The location (directory path) and name of the fields file.

The default setting, if you don’t provide a -f option, is fields.conf in the current directory.

If the fields file, or a file referenced by the fields file, uses the environment variable CONFIG_BASE, then you must enable the Deployment Framework’s command-line completion extension. See Setting up dfw command completion.

--print-info

Displays the version, type and source of the log. The timezone reported is that of the local machine that the logs were written on.

Example:

Logcat: Log Type 'packet' Version 5 created by 'rttpd' in timezone 'Europe/London'

-l

--print-flag-names

Displays the flags that have been sent with each data object; for example F_NONACTIVE|F_CREATEPARENT|F_CREATEOBJECT

For an explanation of what each flag means, see the Data Object Flags module of the DataSource C (DSDK) API Documentation.

-t packet

--log-type packet

Tells logcat to interpret the supplied file as a particular type.

This option takes an argument. Currently the only argument available is packet which makes logcat interpret the file as a packet log. Normally logcat determines the type of the binary log file by looking at a header record within the file, However, you’ll need to use the -t option if you’ve split the file into smaller pieces, because the header will be missing from the smaller file. See Splitting packet logs into smaller files, below.

Example: logcat -t packet mypacket.log

-U <status-file>

--update-file <status-file>

Specifies the name (and optionally the path) of a file to which logcat will write update rate statistics, based on an analysis of the supplied packet log file.

--log-version <version-num>

Specifies the version of the log file, which is then checked against the log versions that this instance of logcat can handle.

This option only has an effect when you pipe the log file to logcat from another tool, such as tail

For example: tail -f <packet-log-file> | logcat --log-version 7

Since the log file header, which identifies the log version, isn’t usually available to logcat via tail, logcat takes the supplied <version-num> to be the log version instead and formats the file contents accordingly.

If <version-num> is greater than the maximum log file version that this instance of logcat can handle, logcat terminates with a version warning message. So in the above example, if logcat cannot handle log versions greater than 5, it displays the following message and then exits:

Logcat: The Log 'Pipe' is of a type and version (DSBLog, p, 7) that I can't handle

-x

--print-id

Displays the datasrc-id of the peer referred to in a packet.

By default, logcat displays the peer’s name as referred to by it’s datasrc-local-label, if that is available.

-z <timezone>

--timezone <timezone>

Sets all times in the log to the specified timezone. To find the required timezone look in the system folder zoneinfo, typically found at /usr/share/lib/zoneinfo or /usr/share/zoneinfo

In this folder there are sub folders with timezone information in them; for example, if there’s a subfolder called US/Mountain', set the timezone option as -z US/Mountain

Usage examples

These examples are for a Liberator packet log. The current working directory is the Liberator’s var directory, where the packet logs are usually located. The default name of Liberator’s packet log is packet-rttpd.log — see the datasrc-pkt-log configuration item in DataSource peers configuration (part 2).

Example 1

Enter the command below:

../bin/logcat --print-info packet-rttpd.log

The output from the command is shown below:

Logcat: Log Type 'packet' Version 5 created by 'rttpd' in timezone 'Europe/London'

Example 2

Enter the command below:

../bin/logcat packet-rttpd.log

The output from the command is shown below (only the first few lines are shown):

2015/09/22-14:10:57.112 +0100: 0.0.0.0 < PEERINFO 102 transformer1 0  ACTIVE|CONTRIB Handshake data:
datasrc-version=6.2.5-301125, hostname=johne-960, platform=i686-pc-win32, technology=DSDK, transformer-version=6.2.5-301153,
pipeline-module-loaded=true, pipeline-module-version=6.2.5-301153, format-module-loaded=true,
format-module-version=6.2.5-301153, autopub-module-loaded=true, autopub-module-version=6.2.5-301153,
cluster-module-loaded=true
2015/09/22-14:10:57.112 +0100: 127.0.0.1 > PEERINFO 101 liberator1 0  BROADCAST Handshake data:
datasrc-version=6.2.5-301125, hostname=johne-960, platform=i686-pc-win32, technology=DSDK, liberator-version=6.2.6-301198,
liberator-auth-module=openauth
2015/09/22-14:10:57.113 +0100: 127.0.0.1 > HEARTBEAT transformer1 15 0
2015/09/22-14:10:57.114 +0100: 127.0.0.1 < HEARTBEAT transformer1 15 0
2015/09/22-14:11:00.297 +0100: 0.0.0.0 < PEERINFO DemoDataSource1 DemoDataSource-johne-960 0 DemoDataSource1 ACTIVE|CONTRIB
Handshake data: datasrc-version=6.2.5-301125, hostname=johne-960, platform=i686-pc-win32, technology=DSDK
2015/09/22-14:11:00.297 +0100: 127.0.0.1 > PEERINFO 101 liberator1 0  BROADCAST Handshake data:
datasrc-version=6.2.5-301125, hostname=johne-960, platform=i686-pc-win32, technology=DSDK, liberator-version=6.2.6-301198,
liberator-auth-module=openauth
2015/09/22-14:11:00.297 +0100: 127.0.0.1 > HEARTBEAT DemoDataSource1 15 0
2015/09/22-14:11:00.298 +0100: 127.0.0.1 < HEARTBEAT DemoDataSource1 15 0
2015/09/22-14:11:00.392 +0100: 127.0.0.1 < DATAUPDATE2 DemoDataSource1 1 8240 /EXAMPLES/BROADCAST/ABC 222 3
-11001=Value 0.243902 -11002=Value 0.149254 -11003=Value 0.294118
2015/09/22-14:11:00.396 +0100: 127.0.0.1 < DATAUPDATE2 DemoDataSource1 2 8240 /EXAMPLES/BROADCAST/DEF 222 3
-11001=Value 1.#INF00 -11002=Value 0.144928 -11003=Value 0.416667
2015/09/22-14:11:00.397 +0100: 127.0.0.1 < DATAUPDATE2 DemoDataSource1 3 8240 /EXAMPLES/BROADCAST/GHI 222 3
-11001=Value 0.128205 -11002=Value 0.172414 -11003=Value 0.161290
...

The log entry timestamps in the example above have a resolution of one millisecond. DataSource 7.1 applications timestamp packet log entries to a resolution of one microsecond, and logcat 7.1 outputs log entry timestamps with a longer fractional second component (6 digits). For example: 2015/09/22-14:10:57.112578 +0100

Displaying the last part of the log file in real time

You can use the Linux tail command with logcat to display the last part of the log file and update the screen when more data appears.

Example:

tail -f packet-rttpd.log | ../bin/logcat -t packet

Splitting packet logs into smaller files

Packet logs can become very large. To make it easier to view a large packet log, you can split it into smaller files first, using the Linux split command.

Example:

split -b 10m packet.log

This example splits the log into separate files of 10 megabytes each.

The split command can produce a lot of files if you’re not careful with the size parameter.

When you use logcat to view one of the split files (other than the first one), you must tell it that the file is a packet log, because the file’s header will now be missing:

logcat -t packet packet-xab

See also: