View packet logs using logcat

The logs produced by most Caplin Platform components are simple text files that can be viewed using a suitable text display utility or text editor, such as the Linux commands cat, more, and vim. However the packet logs are in binary format and must be viewed using the Caplin logcat utility. Here’s how to do this.

Contents:

Running logcat
Examples
Displaying the last part of the log file in real time
Splitting packet logs into smaller files

Running logcat

There’s a copy of the logcat utility in the bin directory of each installed Liberator, Transformer and Integration Adapter.

In the Caplin Deployment Framework, you’ll find logcat in the following places:

<Framework-root>/servers/Liberator/bin
<Framework-root>/servers/Transformer/bin
<Framework-root>/kits/<Adapter-blade-name>/<Adapter-blade-name>/DataSource/bin

where <Framework-root> is the Deployment Framework’s topmost directory.

You can generally use any instance of logcat in your Framework to view any of the packet logs produced by any component running under the Framework. However, we recommend that you normally use the latest logcat whenever possible; that is, the logcat provided in the latest kit version out of all the component kits you’ve deployed in the Framework.

You use logcat in a similar way to the standard Linux cat command. So, if you’re in the directory where logcat resides, you enter a command of the form:

./logcat [options] <path-and-name-of-binary-log-file>

The logcat utility takes the following options:

logcat option (short form) logcat option (long form) Usage

logcat option (short form)	logcat option (long form)	Usage
`-h`	`--help`	Displays detailed information about the available logcat options.
`-F`	`--print-field-names`	Displays the names of the fields in each log record (the default is to just print the field numbers). The field names are obtained from the component’s fields file (see `-f` option).
`-f <fields-file>`	`--fields-file <fields-file>`	The location (directory path) and name of the fields file. The default setting if you don’t provide a `-f` option is `fields.conf` in the current directory.
[not available]	`--print-info`	Displays the version, type and source of the log. For example: `Logcat: Log Type 'packet' Version 5 created by 'rttpd' in timezone 'Europe/London'` The timezone reported is that of the local machine that the logs were written on.
`-l`	`--print-flag-names`	Displays the flags that have been sent with each data object; for example `F_NONACTIVE\|F_CREATEPARENT\|F_CREATEOBJECT` For an explanation of what each flag means, see the Data Object Flags module of the DataSource C (DSDK) API Documentation.
`-t packet`	`--log-type packet`	Tells `logcat` to interpret the supplied file as a particular type. This option takes an argument. Currently the only argument available is `packet` which makes `logcat` interpret the file as a packet log. Normally `logcat` determines the type of the binary log file by looking at a header record within the file, However, you’ll need to use the `-t` option if you’ve split the file into smaller pieces, because the header will be missing from the smaller file. See Splitting packet logs into smaller files, below. Example: `logcat -t packet mypacket.log`
`-U <status-file>`	`--update-file <stats-file>`	Specifies the name (and optionally the path) of a file to which `logcat` will write update rate statistics, based on an analysis of the supplied packet log file.
[not available]	`--log-version <version-num>`	Specifies the version of the log file, which is then checked against the log versions that this instance of `logcat` can handle. This option only has an effect when you pipe the log file to `logcat` from another tool, such as `tail` For example: `tail -f <packet-log-file> \| logcat --log-version 7` Since the log file header, which identifies the log version, isn’t usually available to `logcat` via `tail`, `logcat` takes the supplied `<version-num>` to be the log version instead and formats the file contents accordingly. If `<version-num>` is greater than the maximum log file version that this instance of `logcat` can handle, `logcat` terminates with a version warning message. So in the above example, if `logcat` cannot handle log versions greater than `5`, it displays the following message and then exits: `Logcat: The Log 'Pipe' is of a newer version (7) than I can handle`
`-x`	`--print-id`	Displays the `datasrc-id` of the peer referred to in a packet. By default, `logcat` displays the peer’s name as referred to by it’s `datasrc-local-label`, if that is available.
`-z <timezone>`	`--timezone <timezone>`	Sets all times in the log to the specified timezone. To find the required timezone look in the system folder `zoneinfo`, typically found at `/usr/share/lib/zoneinfo` or `/usr/share/zoneinfo` In this folder there are sub folders with timezone information in them; for example, if there’s a subfolder called `US/Mountain`', set the timezone option as `-z US/Mountain`

-h

--help

Displays detailed information about the available logcat options.

-F

--print-field-names

Displays the names of the fields in each log record (the default is to just print the field numbers). The field names are obtained from the component’s fields file (see -f option).

-f <fields-file>

--fields-file <fields-file>

The location (directory path) and name of the fields file.

The default setting if you don’t provide a -f option is fields.conf in the current directory.

[not available]

--print-info

Displays the version, type and source of the log.

For example:

Logcat: Log Type 'packet' Version 5 created by 'rttpd' in timezone 'Europe/London'

The timezone reported is that of the local machine that the logs were written on.

-l

--print-flag-names

Displays the flags that have been sent with each data object; for example F_NONACTIVE|F_CREATEPARENT|F_CREATEOBJECT

For an explanation of what each flag means, see the Data Object Flags module of the DataSource C (DSDK) API Documentation.

-t packet

--log-type packet

Tells logcat to interpret the supplied file as a particular type.

This option takes an argument. Currently the only argument available is packet which makes logcat interpret the file as a packet log. Normally logcat determines the type of the binary log file by looking at a header record within the file, However, you’ll need to use the -t option if you’ve split the file into smaller pieces, because the header will be missing from the smaller file. See Splitting packet logs into smaller files, below.

Example:

logcat -t packet mypacket.log

-U <status-file>

--update-file <stats-file>

Specifies the name (and optionally the path) of a file to which logcat will write update rate statistics, based on an analysis of the supplied packet log file.

[not available]

--log-version <version-num>

Specifies the version of the log file, which is then checked against the log versions that this instance of logcat can handle.

This option only has an effect when you pipe the log file to logcat from another tool, such as tail

For example: tail -f <packet-log-file> | logcat --log-version 7

Since the log file header, which identifies the log version, isn’t usually available to logcat via tail, logcat takes the supplied <version-num> to be the log version instead and formats the file contents accordingly.

If <version-num> is greater than the maximum log file version that this instance of logcat can handle, logcat terminates with a version warning message. So in the above example, if logcat cannot handle log versions greater than 5, it displays the following message and then exits:

Logcat: The Log 'Pipe' is of a newer version (7) than I can handle

-x

--print-id

Displays the datasrc-id of the peer referred to in a packet.

By default, logcat displays the peer’s name as referred to by it’s datasrc-local-label, if that is available.

-z <timezone>

--timezone <timezone>

Sets all times in the log to the specified timezone. To find the required timezone look in the system folder zoneinfo, typically found at

/usr/share/lib/zoneinfo or /usr/share/zoneinfo

In this folder there are sub folders with timezone information in them; for example, if there’s a subfolder called US/Mountain', set the timezone option as -z US/Mountain

Examples

These examples are for a Liberator packet log. The current working directory is the Liberator’s var directory, where the packet logs are usually located. The default name of Liberator’s packet log is packet-rttpd.log - (see the datasrc-pkt-log configuration item in DataSource peers configuration (part 2)).

Example 1:

../bin/logcat --print-info packet-rttpd.log

This outputs:

Logcat: Log Type 'packet' Version 5 created by 'rttpd' in timezone 'Europe/London'

Example 2:

../bin/logcat packet-rttpd.log

This outputs the following (only the first few lines are shown):

2015/09/22-14:10:57.112 +0100: 0.0.0.0 < PEERINFO 102 transformer1 0  ACTIVE|CONTRIB Handshake data:
datasrc-version=6.2.5-301125, hostname=johne-960, platform=i686-pc-win32, technology=DSDK, transformer-version=6.2.5-301153,
pipeline-module-loaded=true, pipeline-module-version=6.2.5-301153, format-module-loaded=true,
format-module-version=6.2.5-301153, autopub-module-loaded=true, autopub-module-version=6.2.5-301153,
cluster-module-loaded=true
2015/09/22-14:10:57.112 +0100: 127.0.0.1 > PEERINFO 101 liberator1 0  BROADCAST Handshake data:
datasrc-version=6.2.5-301125, hostname=johne-960, platform=i686-pc-win32, technology=DSDK, liberator-version=6.2.6-301198,
liberator-auth-module=openauth
2015/09/22-14:10:57.113 +0100: 127.0.0.1 > HEARTBEAT transformer1 15 0
2015/09/22-14:10:57.114 +0100: 127.0.0.1 < HEARTBEAT transformer1 15 0
2015/09/22-14:11:00.297 +0100: 0.0.0.0 < PEERINFO DemoDataSource1 DemoDataSource-johne-960 0 DemoDataSource1 ACTIVE|CONTRIB
Handshake data: datasrc-version=6.2.5-301125, hostname=johne-960, platform=i686-pc-win32, technology=DSDK
2015/09/22-14:11:00.297 +0100: 127.0.0.1 > PEERINFO 101 liberator1 0  BROADCAST Handshake data:
datasrc-version=6.2.5-301125, hostname=johne-960, platform=i686-pc-win32, technology=DSDK, liberator-version=6.2.6-301198,
liberator-auth-module=openauth
2015/09/22-14:11:00.297 +0100: 127.0.0.1 > HEARTBEAT DemoDataSource1 15 0
2015/09/22-14:11:00.298 +0100: 127.0.0.1 < HEARTBEAT DemoDataSource1 15 0
2015/09/22-14:11:00.392 +0100: 127.0.0.1 < DATAUPDATE2 DemoDataSource1 1 8240 /EXAMPLES/BROADCAST/ABC 222 3
-11001=Value 0.243902 -11002=Value 0.149254 -11003=Value 0.294118
2015/09/22-14:11:00.396 +0100: 127.0.0.1 < DATAUPDATE2 DemoDataSource1 2 8240 /EXAMPLES/BROADCAST/DEF 222 3
-11001=Value 1.#INF00 -11002=Value 0.144928 -11003=Value 0.416667
2015/09/22-14:11:00.397 +0100: 127.0.0.1 < DATAUPDATE2 DemoDataSource1 3 8240 /EXAMPLES/BROADCAST/GHI 222 3
-11001=Value 0.128205 -11002=Value 0.172414 -11003=Value 0.161290
...

Displaying the last part of the log file in real time

You can use the Linux tail command with logcat to display the last part of the log file and update the screen when more data appears.

Example:

tail -f packet-rttpd.log | ../bin/logcat -t packet

Splitting packet logs into smaller files

Packet logs can become very large. To make it easier to view a large packet log, you can split it into smaller files first, using the Linux split command.

Example:

split -b 10m packet.log

This example splits the log into separate files of 10 megabytes each.

The split command can produce a lot of files if you’re not careful with the size parameter.

When you use logact to view one of the split files (other than the first one), you must tell it that the file is a packet log, because the file’s header will now be missing:

logcat -t packet packet-xab

See also:

How can I… Disable packet logging