Search functionality

Ditto provides a search service that lets you query across all managed digital twins using RQL expressions.

Overview

You can access the search functionality through three APIs:

How it works

Search index

Ditto’s things-search microservice automatically consumes all events emitted for changes to Things and Policies. It updates a search-optimized representation in its own database.

You do not need to define custom indexes. The database structure is “flattened” so that all data in Things can be searched efficiently.

Consistency

The search index provides eventual consistency. Updates are written at a default interval of 1 second (configurable via THINGS_SEARCH_UPDATER_STREAM_WRITE_INTERVAL).

When you update a Thing and receive a success response, the search index does not reflect that change immediately. The change typically appears within 1-2 seconds.

If you need to know when a modification is reflected in the search index, request the built-in acknowledgement search-persisted in the command. A status code of 204 confirms successful index update. Status codes at or above 400 indicate failure.

Search queries

You formulate search queries using the Ditto-supported subset of RQL. Queries work through the HTTP API or the Ditto Protocol (e.g., via WebSocket).

Querying scalar JSON values

The query property can target scalar JSON values: booleans, numbers, or strings.

Example – find all Things located in “living-room”, sorted ascending by Thing ID, returning the first 5 results:

Filter:     eq(attributes/location,"living-room")
Sorting:    sort(+thingId)
Paging:     size(5),cursor(CURSOR_ID)

Querying JSON arrays

The query property can also target JSON arrays. Ditto indexes all values in the array, including mixed types.

Given a Thing with tags of different types:

{
  "thingId": "org.eclipse.ditto:tagged-thing-1",
  "policyId": "org.eclipse.ditto:tagged-thing-1",
  "attributes": {
    "tags": [
      "misc", "no-due-date", "high-priority",
      2, 3, 5, false,
      { "room": "kitchen", "floor": 2 }
    ]
  }
}

You can query against scalar values in the array:

eq(attributes/tags,"high-priority")
-> match:       "high-priority" is contained

ne(attributes/tags,"high-priority")
-> no match:    "high-priority" is contained, so "ne" will not match

in(attributes/tags,"misc","something-non-matching")
-> match:       "misc" is a match

like(attributes/tags,"*-priority")
-> match:       "high-priority" string matches

ne(attributes/tags,1)
-> match:       as 1 is not part of the tags

gt(attributes/tags,6)
-> no match:    as no number > 6 is contained

You can also query JSON objects inside the array:

exists(attributes/tags/room)            -> match
eq(attributes/tags/room,"kitchen")      -> match
ge(attributes/tags/floor,2)             -> match

Search count queries

Count queries use the same filter syntax. Sorting and paging options are not applicable for count queries.

Namespaces

You can restrict the search to specific namespaces. This can significantly improve performance when many Things from different namespaces exist in the index.

RQL

Ditto uses RQL notation for search queries and other scenarios such as filtering notifications.

Sorting and paging options

Sorting

The sort option controls the order of search results:

sort(<+|-><property1>,<+|-><property2>,...)

If not specified, results are sorted ascending by Thing ID: sort(+thingId).

Paging with cursor

The size option limits results per response:

size(<count>)

Default: 25. Maximum: 200.

Use cursor-based paging to iterate through large result sets:

cursor(<cursor-id>)

The cursor ID comes from the cursor field of a previous response and marks the position after the last returned entry. No cursor in the response means no more results.

If a request includes a cursor, the filter and sort options must match the original request that produced the cursor.

Example – return ten items with a cursor:

option=size(10),cursor(<cursor-from-previous-result>)

RQL paging (deprecated)

The limit option specifies which page to return:

limit(<offset>,<count>)

Default: limit(0,25). Maximum count: 200.

Example – return the first ten items:

limit(0,10)

Example – return items 11 to 20:

limit(10,10)

Further reading

• RQL expressions – query language reference
• Search protocol – reactive-streams search via Ditto Protocol
• HTTP search API – REST-based search