Ditto 2.4.0 is API and binary compatible to prior Eclipse Ditto 2.x versions.
Changelog
Eclipse Ditto 2.4.0 includes the following topics/enhancements:
- W3C WoT (Web of Things) integration
- SSE (ServerSentEvent) API for subscribing to messages
- Recovery status for connections indicating when e.g. recovery is no longer tried after max backoff
- Enhance placeholders to resolve to multiple values
- Advanced JWT placeholder operations
- Support for a wildcard/placeholder identifying the changed feature in order to enrich e.g. its definition
The following notable fixes are included:
- Several fixes and improvements regarding consistency and performance of search updates
- Don’t publish messages with failed enrichments and issue failed ack
- Filter for incorrect element types in jsonArray of feature definitions
- Fix of placeholder resolvment in “commandHeaders” of “ImplicitThingCreation” mapper
- Fix
fn:substring-after()
function returning incorrect data
The following non-functional work is also included:
- Upgrade of compiler target level for service modules from Java 11 to Java 17
- Switch of used Java runtime in pre-built Docker containers from OpenJ9 to Hotspot
- Publication of pre-built multi-architecture Docker images for
linux/amd64
(as always) and now in additionlinux/arm64
- Removal of rate limiting / throttling limits as default
- Update of several used dependencies
For a complete list of all merged PRs, inspect the following milestones:
Compared to the latest release 2.3.0, the following most notable changes, new features and bugfixes were added:
Changes
Upgrade to Java 17 + change of Java runtime to Hotspot
We upgraded the compiler target level for our service modules from 11 to 17 and also use a Java 17 runtime environment
for our service containers. Please note that the Ditto model still remains compatible to Java 8.
This change only affects you when you’re building and deploying Ditto on your own.
Publication of pre-built multi-architecture Docker images
Acknowledging the raise of the ARM processor architecture, starting with Ditto 2.4.0, Docker images for the following architectures will be published to docker.io:
linux/amd64
linux/arm64
(new)
Removal of rate limiting / throttling limits as default
By default, Ditto had configurations in place to rate limit consumption of messages received via:
- AMQP 1.0 connections
- Apache Kafka connections
- the WebSocket endpoint
These limitations are now by default disabled (as they mainly make sense for a multi-tenant environment) and can be enabled manually, as mentioned in the configuration - rate limiting section.
New features
W3C WoT (Web of Things) integration
Ditto adds and optional (and currently experimental) integration of the Web of Things (WoT) Thing Description 1.1 specification.
Digital twins managed by Ditto can reference to WoT “Thing Models” (being accessible via an HTTP URL) in which the
capabilities of the twin are defined.
Using this linked model, Ditto can generate a WoT “Thing Description” on a thing instance level and e.g. generate a JSON
skeleton upon creation of a thing.
For more details, please have a look at the blogpost and the
WoT integration documentation.
SSE (ServerSentEvent) API for subscribing to messages
Messages to or from a digital twin can now be subscribed to with the SSE endpoint, either on Thing level or for a specific Feature
Recovery status for connections indicating when e.g. recovery is no longer tried after max backoff
The new recovery status contains one of the values:
- ongoing
- succeeded
- backOffLimitReached
- unknown
and can be used to find out whether an automatic failover is still ongoing or if the max amount of configured reconnects applying backoff was reached and that recovery is no longer happening.
Enhance placeholders to resolve to multiple values
Placeholders may now resolve to multiple values instead of only a single one which enables e.g. applying placeholder functions to each element of an array.
Advanced JWT placeholder operations
Using the above feature of placeholders being resolved to multiple values, the JWT placeholder, which can be used
in scope of the OpenID connect configuration, can now
be used with functions.
This can e.g. be used in order to filter out unwanted subjects in the
OpenId connect configuration, or to additionally split a JWT claim into
several values using the new fn:split(' ')
function.
Example extracting only subjects from a JSON array “roles” contained in a JWT ending with “moderator”:
{{ jwt:extra/roles | fn:filter('like','*moderator') }}
Support for a wildcard/placeholder identifying the changed feature in order to enrich e.g. its definition
Using the above feature of placeholders being resolved to multiple values, it is now possible to use a placeholder
{{ feature:id }}
as part of an enrichment extraFields
pointer resolving
to all affected feature ids of a change.
This can e.g. be used to enrich all the feature definition
s of all modified features in an event which should be
published via websocket or a connection:
extraFields=features/{{feature:id}}/definition
Bugfixes
Several bugs in Ditto 2.3.x were fixed for 2.4.0. This is a complete list of the merged pull requests for 2.4.0-M1 and merged pull requests for 2.4.0, including the fixed bugs.
Migration notes
Migrations required updating from Ditto 2.3.x or earlier versions:
- With this release we not only switched from Java 11 to Java 17 but also from OpenJ9 to Hotspot runtime.
This means that the environment variable
OPENJ9_JAVA_OPTIONS
needs to be renamed toJAVA_TOOL_OPTIONS
and that options specific to the OpenJ9 runtime are no longer effective.
Make sure that all options that are defined are valid for the Hotspot JVM.
Ditto clients
For a complete list of all merged client PRs, inspect the following milestones:
Ditto Java SDK
Fix returning the revision of a Policy when retrieved via the client
When using the client.policies().retrievePolicy(PolicyId)
functionality in the Ditto Java client, the getRevision()
method of the returned policy was always empty.
The revision will now be included.
Ditto JavaScript SDK
See separate Changelog of JS client.
Roadmap
Looking forward, the plan for Ditto 2.5.0 is to work on:
- Enhancing MQTT (3.1.1 and 5) connections to use reactive MQTT driver (with backpressure support)
- Improving of MongoDB based search index used in things-search,
using a new index structure:
- smaller index size for large thing / policy combinations
- fewer used CPU resources on MongoDB
- faster search queries
- better vertical scalability of search index
Looking even more ahead, the plan for Ditto 3.0.0 is to work on:
- Simplifying Ditto’s architecture by:
- removing the “concierge service”
- potentially (to be evaluated) even removing the gateway service and merging its functionality (providing HTTP + WS endpoints) into the connectivity service
- Finalizing the experimental WoT (Web of Things) integration when the “Thing Description 1.1” is published as W3C Recommendation