Structure of JSON Event Files

Introduction

This document describes the format of the JSON event files, which are indicated by file extension .jsonl.

Events are commands, signals, notifications and replies to commands. They are defined in a CommaSuite interface signature. The JSON event files contain a collection of events in the order of their execution with respect to the defined model.

The target audience are engineers that implement event file generators. Event files are not intended to be manually constructed by the users. Usually they are automatically generated from existing system logs and observed client-server communication. The file structure is optimized for machine processing.

Events in JSON Format

Events are written in JSON files with extension .jsonl and they have a different format than the events written in files with .events extension. The events still have the same CommaSuite meaning and rules applied as described in events files, only the structure will be expressed in JSON format. For illustration, the following example trace written in .events file is used to show how this is mapped to JSON format :

import "../IImaging.signature"
import "../ITemprature.signature"
import "../IVacuum.signature"
import "../Imaging.component"

connections
(Client1, p1, IImaging, 0 c0, iImagPort)
(Client2, p2, ITemprature, 0 c0, iTempPort)
(Client3, p3, IVacuum, 0 c0, iVacPort)

components
Imaging c0

events

command 1.0 0.0 Client2 p2 c0 iTempPort ITemperature SetTemperature
int 80
End

Reply 1.002 0.002 c0 iTempPort Client2 p2 ITemperature SetTemperature
End

Events are expressed as JSON elements in a JSON file. Each event (command, signal, notification and reply) is written as a JSONObject. A JSONObject contains a set of objects in the form of a {key:value} pairs. The JSON trace file consists of two types of JSONObjects:

  • Optional declaration of component instances

  • Events

Component instances

A component instance is created only when a monitor task is created for a component. An instance of a component is expressed as a JSONObject that exists of the following keys:

  • components : A list of component instances.

A component instance contains the following keys:

  • type : Component model.

  • value : Name of the instance of the component model. The name value must be of type string.

As an example the component instance c0 of the example trace above is expressed in JSON format as follows:

component
{"components" : [{"type":"Imaging","value":"c0"}]}

The structure of a component instance in JSON format is as follows:

{"components" : [{"type":"Name of component model","value":"instance of the component model"}]}
Components instances are always declared first in a JSON file. The events are declared after the declaration of component instances. Connections and imports are not used in JSON files, although they are required in the .events files.

Events

Events (commands, signal, notification and reply) have more keys than component JSONObjects. A JSONObject event has the following collection of keys:

  • kind : Kind of event (Command, Signal, Notification or Reply)

  • timeStamp : Time of execution. Denoted as a string in ISO 8601 format YYYY-MM-DD’T’hh:mm:ss.ddd'+|-'hh:mm . See, for instance, Wikipedia on ISO 8601.

    • Although the standard allows several notations, in CommaSuite it is restricted to the template above. An example: "2023-08-02T13:55:36.138+02:00".

    • Note that 16:38:26.89 is interpreted by the ComMA monitor as 16:38:26.089 (and not as 16:38:26.890).

  • source : The source of the event

  • sourcePort : The communication port of the source

  • destination : The destionation of the event

  • destinationPort : The communication port of the destination

  • interface : The name of the interface model

  • method : Event name as defined in the interface model

  • parameters : A list of parameters of the event

Optional
  • metadata : A JSONObject with additional information about the event. CommaSuite does not use this object; it can be exploited for other purposes. For instance, the meta data may contain information about the log from which the event was extracted which can be useful to debug errors.

In general an event is represented by a JSONObject with the following structure:

{"kind":"..","timeStamp":"..","source":"..","sourcePort":"..","destination":"..","destinationPort":"..",  "interface":"..","method":"..","parameters":[]}
The required keys (kind, timeStamp,..) of the objects have to be present in the JSONObject event. The key parameters can be left out if there are no parameters declared in the event.
The parameters expects a list of parameters; all other objects require a value of type string.

As an illustration, the events of the example trace will be denoted in JSON. The the component instance c0 has already been described above. The events (command and reply) are represented as JSONObject as follows:

Command event
{"kind": "Command", "timeStamp": "2021-02-16T20:22:28.000+00:00","interface": "ITemperature", "source":"Client2", "sourcePort": "p2", "destination":"c0", "destinationPort": "iTempPort","method": "SetTemprature", "parameters": [{"type":"int","value":80}]}
Reply event
{"kind": "Reply", "timeStamp": "2021-02-16T20:22:28.000+00:00","interface": "ITemperature", "source":"c0", "sourcePort": "iTempPort", "destination":"Client2", "destinationPort": "p2","method": "SetTemperature"}

Parameters

An event may have zero or more parameters. If a command or a reply has parameters, they are passed as a list to the key parameters. A command event gives its in and inout parameters in the order of their definition in the signature.

A reply to a command gives parameters in the following order: first inout and out parameters in the order of their definition, if any, and next the reply value.

CommaSuite has a set of data types to indicate the type of a parameter.

  • Simple types: int, bool, real, string, bulkdata

  • Enum types : enum

  • Record types: record

  • Collection types: vector

Expressing simple parameters in JSON format

The parameter values of simple types and enums are expressed in JSON format as follows.

Simple types:

  • int: 1, 123

  • bool: true, false

  • real: 123.0, NaN

  • string: “some string”

  • bulkdata: the value of a bulkdata is just an integer that indicates the number of bytes in the data. Important: the actual data are not given.

Enum types: the name of the type and the literal, e.g. "Status::OK"

Every simple parameter is expressed in a JSON format using the keys type and value as follows:

{"type":simple types,"value":simple values}

In the example trace described above, the event command has a simple parameter of type int with value 80, which is expressed in JSON format as

{"type":"int","value":80}

This parameter is included in the list of parameters:

{..,"parameters":[{"type":"int","value":80}]}

Examples of other simple types:

{"type":"real","value":80.34}
{"type":"bool","value":true}
{"type":"bulkdata","value":10}
{"type":"enum","value":"Status::OK"}
Vector values

The parameter vector has the following three keys :

The typeElem key describes the data type of the elements of the vector. The key value contains the actual elements as a list.

So a vector parameter is expressed in a JSON format as follows :

{"type":"vector", typeElem:type_indicators, "value":[parameter(s)]}

For example, vectorInt is a Vector type with elements of simple type int and intValues is a variable of this type. Similarly, realVector contains elements of the simple type real and realValues is a variable of this type. Both variables are defined and initialized as follows:

intValues = <vectorInt>[1,3,5,7,11]
realValues = <vectorReal>[8.3, 6.7, 9.0, 11.3]

These vector values are denoted in JSON format as follows:

{"type":"vector", typeElem:"int", "value":[1,3,5,7,11]}
{"type":"vector", typeElem:"real", "value":[8.3,6.7,9.0,11.3]}

If typeElem is a simple parameter , then values are not passed as a parameter object {"type":"","value":""} to the key value, but only the actual values are passed, as shown in the two vectors examples above. This does not hold for the parameters of type Record and Vector as shown in the next example of a multidimensional vector.

Consider a matrix, which is a 2-dimensional vector type with elements of type int. A value can be assigned to a variable matrixValues of this type as follows:

matrixValues := <matrix>[<int[2]>[1, 2], <int[2]>[3, 4]]

This matrix value is represented in JSON format as follows:

{"type":"vector","typeElem":"vector", "value":[{"type":"vector",typeElem:"int", "value":[1,2]},{"type":"vector",typeElem:"int" "value":[3,4]}]}
Record values

The JSON format of a parameter of type record contains three keys:

  • type : record

  • record : the name of the record as defined in the interface

  • value : contains the fields of the record with their assigned values

In general, the structure of a parameter record is as follows :

{"type":"record", "record":"name of record", "value":{"field1":"value","field2":"value",..}}

For example, consider a record EnergySupplier defined as:

record EnergySupplier {
string name
real price
}

An instance of record EnergySupplier is denoted as follows:

{"type":"record", "record":"EnergySupplier", "value":{"name":"Eneco","price":81.9}}

An example of a nested record:

record Location {
string street
string place
}

record EnergySupplier {
string name
real price
Location location
}

An instance is denoted in JSON as follows:

{"type":"record", "record":"EnergySupplier", "value":{"name":"Eneco","price":81.9,"location":{"type":"record","record":"Location","value":{"street":"kruisstraat","place":"eindhoven"}}}}
The JSON file structure

A JSON file (extension .jsonl) optionally starts with component instances and next contains a collection of JSONObjects where each object represents an event. Each event has to be placed on a separate line.

{"components" : [{type":"Name of component model","value":"instance of the component model"}]}
{"kind":"", "timeStamp":"..", "source":"..",..}
{"kind":"", "timeStamp":"..",  "source":"..",..}
{"kind":"", "timeStamp":"..",  "source":"..",..}
{...}
...

Every event has to be placed on a single line as a JSONObject. It is not allowed to place parts of the event on multiple lines.

The CommaSuite JSON files use the extension .jsonl (and not the general JSON extension .json) to indicate that these files contain new lines.

Small examples can be found in the JSON Events Example; the Sensor Server Example shows a few long JSON event files of a component.

Issues with Large Event Files

Opening a trace file in the Eclipse IDE will cause parsing of the text and the execution of the validation checks. If the file is very large (most realistic event files are large) an 'out of memory' exception may occur. Hence it is not recommended to inspect and edit such files in the Eclipse editor.

Reading large events files during monitoring (either invoked from the Eclipse IDE or from the command line) is not a problem because the monitor reads the file incrementally in order to reduce memory usage.