Ditto’s HTTP API follows some concepts which are documented on this page.
The entry point into the HTTP API is:
http://localhost:8080/api/<apiVersion>
API versioning
Ditto’s HTTP API is versioned in the URL: /api/<apiVersion>
. Currently, Ditto only provides API version 2
.
API version 1 was deprecated and deleted as of Ditto version 2.0.0
The API version is a promise that no HTTP resources (the static ones defined by Ditto itself) are modified in an
incompatible/breaking way. As the HTTP resources reflect the JSON structure of the Thing
entity, that also applies
for this entity.
In API 2 the Thing
structure contains a Policy where the authorization information is
managed.
Endpoints
In the HTTP API, some endpoints are static and can be seen as the “schema” of Ditto. They are in sync with the JSON
representation of the model classes, e.g. Thing for the layout of the /things
endpoint and Policy for the layout of the /policies
endpoint.
API version 2
In API version 2, a Thing
contains a policyId
, which points to a Policy
managed as another entity.
Its API endpoint is /policies
.
/things
in API 2
The base endpoint for accessing and working with Things
.
A Thing
in API 2 has the following JSON structure:
{
"thingId": "{thingId}",
"policyId": "{policyId}",
"definition": "{definition}",
"attributes": {
},
"features": {
}
}
This maps to the following HTTP API endpoints:
/things/{thingId}
: accessing a complete specific thing/things/{thingId}/policyId
: accessing the policy ID of the specific thing/things/{thingId}/definition
: accessing the definition of the specific thing/things/{thingId}/attributes
: accessing the attributes of the specific thing/things/{thingId}/features
: accessing the features of the specific thing
/things
in API 2 - dynamic part
Additionally, to that “static part” of the HTTP API which is defined by Ditto, the API is dynamically enhanced by the
JSON structure of the Thing.
For example for a Thing
with following content:
{
"thingId": "{thingId}",
"policyId": "{policyId}",
"definition": "{definition}",
"attributes": {
"manufacturer": "ACME corp",
"complex": {
"some": false,
"serialNo": 4711
}
},
"features": {
"lamp": {
"properties": {
"on": false,
"color": "blue"
}
}
}
}
The following additional API endpoints are automatically available:
/things/{thingId}/attributes/manufacturer
: accessing the attributemanufacturer
of the specific thing/things/{thingId}/attributes/complex
: accessing the attributecomplex
of the specific thing/things/{thingId}/attributes/complex/some
: accessing the attributecomplex/some
of the specific thing/things/{thingId}/attributes/complex/serialNo
: accessing the attributecomplex/serialNo
of the specific thing/things/{thingId}/features/lamp
: accessing the featurelamp
of the specific thing/things/{thingId}/features/lamp/properties
: accessing all properties of the featurelamp
of the specific thing/things/{thingId}/features/lamp/properties/on
: accessing theon
property of the featurelamp
of the specific thing/things/{thingId}/features/lamp/properties/color
: accessing thecolor
properties of the featurelamp
of the specific thing
/policies
in API 2
The base endpoint for accessing and working with Policies
.
A Policy
in API 2 has the following JSON structure:
{
"policyId": "{policyId}",
"entries": {
"{entryLabel-1}": {
"subjects": {
"{subjectId1}": {
}
},
"resources": {
"{resource1}": {
}
}
}
}
}
This maps to the following HTTP API endpoints:
/policies/{policyId}
: accessing completePolicy
/policies/{policyId}/entries
: accessing thePolicy
entries/policies/{policyId}/entries/{entryLabel-1}
: accessing a singlePolicy
entry with the label{entryLabel-1}
/policies/{policyId}/entries/{entryLabel-1}/subjects
: accessing the subjects of a singlePolicy
entry with the label{entryLabel-1}
/policies/{policyId}/entries/{entryLabel-1}/resources
: accessing the resources of a singlePolicy
entry with the label{entryLabel-1}
/connections
in API 2
The base endpoint for accessing and working with Connections
.
It has the following HTTP API endpoints:
/connections
: accessing all connections or create a new connection/connections/{connectionId}
: accessing a specificConnection
entry/connections/{connectionId}/command
: send a command to a specific Connection/connections/{connectionId}/status
: retrieve status of a specific Connection/connections/{connectionId}/metrics
: retrieve metrics of a specific Connection/connections/{connectionId}/logs
: retrieve logs of a specific Connection
Partial updates
As a benefit of the above-mentioned mechanism that an API is automatically available based on the JSON structure, the “partial update” pattern can be applied when modifying data.
The benefit of this is a reduction in payload to be transferred. Further, it is beneficial because other parts of the
Thing
are not overwritten with a potentially outdated value - only the actually changed data part can be modified.
So instead of modifying a complete Thing
only a specific part is affected.
Given, the on
property of lamp
should be changed to true
.
Instead of
PUT .../things/{thingId}
with the complete payload:
{
"thingId": "{thingId}",
"policyId": "{policyId}",
"definition": "{definition}",
"attributes": {
"manufacturer": "ACME corp",
"complex": {
"some": false,
"serialNo": 4711
}
},
"features": {
"lamp": {
"properties": {
"on": true,
"color": "blue"
}
}
}
}
we can use a smarter requestPUT .../things/{thingId}/features/lamp/properties/on
with a minimal payload:
true
Partial requests
Similar to the partial updates from above, the HTTP API can also be used to retrieve a single value instead of a
complete Thing
.
Again, the benefit is a reduction in response payload and that the caller can directly use the returned data value
(for example expect it to be a boolean
and treat it accordingly).
For example, we can requestGET .../things/{thingId}/features/lamp/properties/on
and get as response:
true
With field selector
A further mechanism in the API for partial requests is using a so-called field selector. This is useful when the JSON
structure of the Thing
or other entity should be kept intact, but not all information is relevant.
The field selector is passed as an HTTP query parameter fields
and contains a comma separated list of fields to include
in the response.
It is possible to use the wildcard operator ‘*’ as feature ID and retrieve a property of multiple features. For details see the example below.
Given, you have the following Thing:
{
"thingId": "{thingId}",
"policyId": "{policyId}",
"definition": "{definition}",
"attributes": {
"manufacturer": "ACME corp",
"complex": {
"some": false,
"serialNo": 4711,
"misc": "foo"
}
},
"features": {
"lamp": {
"properties": {
"on": true,
"color": "blue"
}
},
"infrared-lamp": {
"properties": {
"on": false,
"color": "red"
}
}
}
}
Field selector examples
The following GET
request examples with field selectors show how you can retrieve only the parts of a thing which you
are interested in:
GET .../things/{thingId}?fields=attributes
Response:
{
"attributes": {
"manufacturer": "ACME corp",
"complex": {
"some": false,
"serialNo": 4711,
"misc": "foo"
}
}
}
GET .../things/{thingId}?fields=attributes/manufacturer
Response:
{
"attributes": {
"manufacturer": "ACME corp"
}
}
GET .../things/{thingId}?fields=attributes/complex/serialNo
Response:
{
"attributes": {
"complex": {
"serialNo": 4711
}
}
}
GET .../things/{thingId}?fields=attributes/complex/some,attributes/complex/serialNo
Response:
{
"attributes": {
"complex": {
"some": false,
"serialNo": 4711
}
}
}
GET .../things/{thingId}?fields=attributes/complex(some,serialNo)
Response:
{
"attributes": {
"complex": {
"some": false,
"serialNo": 4711
}
}
}
GET .../things/{thingId}?fields=attributes/complex/misc,features/lamp/properties/on
Response:
{
"attributes": {
"complex": {
"misc": "foo"
}
},
"features": {
"lamp": {
"properties": {
"on": true
}
}
}
}
Field selector with wildcard
GET .../things/{thingId}?fields=features/*/properties/on
Response:
{
"features": {
"lamp": {
"properties": {
"on": true
}
},
"infrared-lamp": {
"properties": {
"on": false
}
}
}
}
Merge updates
Merge updates can be used to update multiple parts of a Thing in a single request e.g. multiple properties of
different features or a feature property and an attribute value. Merge updates are applied by using the HTTP
PATCH
method with the payload in JSON merge patch (RFC-7396) format. The
content-type of the request must be set to application/merge-patch+json
.
RFC-7396 specifies how a set of modifications is applied to an existing JSON document:
A JSON merge patch document describes changes to be made to a target JSON document using a syntax that closely
mimics the document being modified. Recipients of a merge patch document determine the exact set of changes being
requested by comparing the content of the provided patch against the current content of the target document.
If the provided merge patch contains members that do not appear within the target, those members are added. If the
target does contain the member, the value is replaced. Null values in the merge patch are given special meaning to
indicate the removal of existing values in the target.
null
values. When using PATCH
a null
value is
interpreted as delete in contrast to PUT
requests where null
values have no special meaning. Like PUT
requests, PATCH
requests can be applied at any level of the JSON structure of a thing, e.g. patching a
complete thing at root level or patching a single property value at property level.
Removing fields in a merge update with a regex
null
values
for deleting certain parts of JSON objects specified with a regular expression before applying new fields to it.The merge patch functionality in Ditto solves a common problem to the JSON merge patch (RFC-7396): whenever a JSON object
shall be patched, all the old json fields are merged with all the new json fields, unless the exact field names are
specified in the patch with "<field>": null
.
This would however require to know all existing fields upfront, which for a merge patch can not be assumed.
The solution is a little enhancement to Ditto’s merge patch functionality: The ability to delete arbitrary parts from JSON objects using a regular expression before applying all other patch values.
The syntax for this function is rather specific (so that no “normally” occurring JSON keys match the same syntax):
{
"{{ ~.*~ }}": null
}
The recommended regex delimiter to use is the ~
character, additionally supported delimiters at this point are:
~
/
(discouraged due to the special meaning of/
for HTTP paths)
A discouraged and thus deprecated way to define the regex (using /
as delimiter) is:
{
"{{ /.*/ }}": null
}
When such a {{ ~<regex>~ }}
with the value null
is detected in the merge patch, the content between the 2 delimiters is
interpreted as regular expression to apply for finding keys to delete from the target object.
As a result, using "{{ ~.*~ }}": null
would delete all the values inside a JSON object before applying the new
values provided in the patch.
Example:
Assuming that inside a JSON object every month some aggregated data is stored with the year and month:
{
"thingId": "{thingId}",
"policyId": "{policyId}",
"features": {
"aggregated-history": {
"properties": {
"2022-11": 42.3,
"2022-12": 54.3,
"2023-01": 80.2,
"2023-02": 99.9
}
}
}
}
Then the data from “last year” could be purged with the following patch, while adding a new value to the existing ones of this year:
{
"features": {
"aggregated-history": {
"properties": {
"{{ ~2022-.*~ }}": null,
"2023-03": 105.21
}
}
}
}
The resulting Thing JSON after applying the patch would then look like:
{
"thingId": "{thingId}",
"policyId": "{policyId}",
"features": {
"aggregated-history": {
"properties": {
"2023-01": 80.2,
"2023-02": 99.9,
"2023-03": 105.21
}
}
}
}
Permissions required for merge update
To successfully execute merge update the authorized subject needs to have WRITE permission on all resources affected by the provided JSON merge patch. If the permission is missing for one of the affected resources the whole merge patch is rejected, i.e. the merge update is executed as a whole or not at all.
Merge update example
Given an existing thing with the JSON structure:
{
"thingId": "{thingId}",
"policyId": "{policyId}",
"attributes": {
"location": {
"longitude": 47.682170,
"latitude": 9.386372
},
"serialNo": "0000000"
},
"features": {
"temperature": {
"properties": {
"value": 25.43,
"unit": "°C"
}
},
"pressure": {
"properties": {
"value": 1013.25,
"unit": "hPa"
}
}
}
}
Assuming a single request should:
- add the
manufacturer
attribute - update the existing
serialNo
attribute to the value of23091861
- remove the existing
location
attribute - set the existing property
value
of featuretemperature
to the value of26.89
- remove the existing property
unit
of featurepressure
- add a new feature
humidity
This can be achieved using a PATCH .../things/{thingId}
with the request payload of
{
"attributes": {
"location": null,
"manufacturer": "Bosch",
"serialNo": "23091861"
},
"features": {
"temperature": {
"properties": {
"value": 26.89
}
},
"pressure": {
"properties": {
"unit": null
}
},
"humidity": {
"properties": {
"value": 55,
"unit": "%"
}
}
}
}
The resulting JSON representation of the updated thing after applying the PATCH
is:
{
"thingId": "{thingId}",
"policyId": "{policyId}",
"attributes": {
"manufacturer": "Bosch",
"serialNo": "23091861"
},
"features": {
"temperature": {
"properties": {
"value": 26.89,
"unit": "°C"
}
},
"pressure": {
"properties": {
"value": 1015
}
},
"humidity": {
"properties": {
"value": 55,
"unit": "%"
}
}
}
}
Conditional Requests
The HTTP API for Things
and Policies
partially supports Conditional Requests
as defined
in RFC-7232.
ETag
A successful response on a thing
or policy
resource provides an ETag
header.
- For read responses, it contains the current entity-tag of the resource.
- For write responses, it contains the entity-tag after successful write.
The ETag
has a different format for top-level resources and sub-resources.
- Top-level resources (e.g.
.../things/{thingId}
): The entity-tag contains the revision of the entity which is addressed by the resource in the format"rev:<revision>"
, e.g."rev:2"
. - Sub-resources (e.g.
.../things/{thingId}/features/{featureId}
): The entity-tag contains a hash of the current value of the addressed sub-resource in the format"hash:<calculated-hash>"
, e.g."hash:87192253740"
. Note that this format may change in the future.
Conditional Headers
The following request headers can be used to issue a conditional request:
If-Match
:- Read or write the resource only
- if the current entity-tag matches at least one of the entity-tags provided in this header
- or if the header is
*
and the entity exists
- The response will be:
- in case of a match, the same response as if the header wouldn’t have been specified
- in case of no match, status
412 (Precondition Failed)
with an error response containing detail information and the current entity-tag of the resource asETag
header
- Read or write the resource only
If-None-Match
:- Read or write the resource only
- if the current entity-tag does not match any one of the entity-tags provided in this header
- or if the header is
*
and the entity does not exist
- The response will be:
- in case of no match, the same response as if the header wouldn’t have been specified
- in case of a match:
- for write requests, status
412 (Precondition Failed)
with an error response containing detail information and the current entity-tag of the resource asETag
header - for read requests, status
304 (Not Modified)
without response body, with the current entity-tag of the resource asETag
header
- for write requests, status
- Read or write the resource only
if-equal
:- The
if-equal
header can take the values'update'
(which is the default if omitted),'skip'
or'skip-minimizing-merge'
- Modify/Update the resource only
- in case
if-equal: 'update'
is defined always updates - even if the entity is equal before the update. - in case
if-equal: 'skip'
is defined, the entity will not be updated if it is equal before the update.
In this case a ‘Precondition Failed’ 412 status is returned. - in case
if-equal: 'skip-minimizing-merge'
is defined, the entity will not be updated if it is equal before the update.
In this case a ‘Precondition Failed’ 412 status is returned. Additionally, Merge commands will be minimized to only the fields which actually changed, compared to the current state of the entity.- this reduces the part of a merge/patch command to the actually changed elements, removing non-changed elements
- this reduces e.g. required storage in the MongoDB by a lot, if redundant data is sent often
- this also reduces the event payload (e.g. emitted to subscribers) to only the actually changed parts of the thing
- in case
- The
Note that the Ditto HTTP API always provides a strong
entity-tag in the ETag
header, thus you will never receive a
weak
entity-tag (see RFC-7232 Section 2.1). If you convert this
strong entity-tag to a weak entity-tag and use it in a Conditional Header, Ditto will handle it according to RFC-7232.
However, we discourage the usage of weak entity-tags, because in the context of Ditto they only add unnecessary
complexity.
In addition to the ETag
header Ditto supports conditional requests with a condition
header. For further information
see Conditional Requests
Exempted fields
Assuming you have a thing with an associated policy. When querying the thing with
GET .../things/{thingId}?fields=_policy
you will get the thing containing its revision and associated policy.
If you now modify the associated policy, the revision of the thing will not change! This could lead to an inconsistent
state if the thing is getting refetched by using the If-None-Match
header, because this would return
a 304 Not Modified
, even if the policy has changed.
To tackle this, Ditto has the following list of exempted fields which automatically bypass the precondition header check:
_policy
Examples
The following examples show several scenarios on a top-level (Thing) resource. Nevertheless, these scenarios can also be applied on any sub-resource in the same way.
Create: Write only if the resource does not exist
The following example request shows, how you can make sure that a PUT
request does not overwrite existing data, i.e.
how you can enforce that the Thing can only be created by the request.
PUT .../things/{thingId}
If-None-Match: *
{
"policyId": "{policyId}",
"attributes": {
"manufacturer": "ACME crop",
"otherData": 4711
}
}
You will get one of the following responses:
201 (Created)
in case the creation was successful, i.e. the Thing did not yet exist.412 (Precondition Failed)
in case the creation failed, i.e. a Thing with the exactly same{thingId}
already exists.
Update: Write only if the resource already exists
The following example request shows how you can make sure that a PUT
request does not create the resource, i.e. how
you can enforce that the Thing can only be updated by the request, but you do not generate a duplicate by mistake.
PUT .../things/{thingId}
If-Match: *
{
"attributes": {
"manufacturer": "ACME crop",
"otherData": 4711
}
}
You will get one of the following responses:
204 (No Content)
in case the update was successful, i.e. the Thing already existed.412 (Precondition Failed)
in case the update failed, i.e. the Thing does not yet exist.
Optimistic Locking
First, GET
the Thing in order to retrieve both: the current data and the entity-tag:
GET .../things/{thingId}
:
Response:
ETag: "rev:2"
{
"thingId": "{thingId}",
"policyId": "{policyId}",
"definition": "{definition}",
"attributes": {
"manufacturer": "ACME crop",
"otherData": 4711
}
}
Assume that you have detected the typo in the manufacturer attribute (“ACME crop”) and want to fix this with a top-level Thing PUT. You want to make sure, that no one else has modified the Thing in the meantime, because otherwise his changes would be lost. (You could also achieve this with a PUT on the concrete attribute, but for this example we assume that you want to use a top-level Thing PUT.)
PUT
the Thing with the changed data and the entity-tag from the preceding GET
response in the If-Match
header.
PUT .../things/{thingId}
If-Match: "rev:2"
{
"attributes": {
"manufacturer": "ACME corp",
"otherData": 4711
}
}
You will get one of the following responses:
204 (No Content)
in case the update was successful, i.e. no one else has changed the Thing in the meantime.412 (Precondition Failed)
in case the update was not successful, i.e. the Thing has been changed by someone else in the meantime.