Configuring the Command Router Service

The Command Router service provides an implementation of Eclipse Hono™’s Command Router API.

Protocol Adapters use the Command Router API to supply information that the Command Router service component can use to route command & control messages to the particular protocol adapter instances that the target devices are connected to.

The Command Router component provides an implementation of the Command Router API which uses a remote data grid for storing information about device connections. The data grid can be scaled out independently from the Command Router service components to meet the storage demands at hand.

The Command Router is implemented as a Quarkus application. It can be run either directly from the command line or by means of starting the corresponding Docker image created from it.

Service Configuration

The following table provides an overview of the configuration variables and corresponding system properties for configuring the Command Router component.

The variables only need to be set if the default value does not match your environment.

In addition to the options described in the table above, this component supports the following standard configuration options:

• Common Java VM Options
• Common vert.x Options
• Monitoring Options

Port Configuration

The Command Router component supports configuration of an AMQP based endpoint that can be configured to listen for connections on

• a secure port only (default) or
• an insecure port only or
• both a secure and an insecure port (dual port configuration)

The server will fail to start if none of the ports is configured properly.

Secure Port Only

The server needs to be configured with a private key and certificate in order to open a TLS secured port.

There are two alternative ways for doing so:

1. Setting the HONO_COMMANDROUTER_AMQP_KEYSTOREPATH and the HONO_COMMANDROUTER_AMQP_KEYSTOREPASSWORD variables in order to load the key & certificate from a password protected key store, or
2. setting the HONO_COMMANDROUTER_AMQP_KEYPATH and HONO_COMMANDROUTER_AMQP_CERTPATH variables in order to load the key and certificate from two separate PEM files in PKCS8 format.

When starting up, the server will bind a TLS secured socket to the default secure AMQP port 5671. The port number can also be set explicitly using the HONO_COMMANDROUTER_AMQP_PORT variable.

The HONO_COMMANDROUTER_AMQP_BINDADDRESS variable can be used to specify the network interface that the port should be exposed on. By default the port is bound to the loopback device only, i.e. the port will only be accessible from the local host. Setting this variable to 0.0.0.0 will let the port being bound to all network interfaces (be careful not to expose the port unintentionally to the outside world).

Insecure Port Only

The secure port will mostly be required for production scenarios. However, it might be desirable to expose a non-TLS secured port instead, e.g. for testing purposes. In any case, the non-secure port needs to be explicitly enabled either by

• explicitly setting HONO_COMMANDROUTER_AMQP_INSECUREPORT to a valid port number, or by
• implicitly configuring the default AMQP port (5672) by simply setting HONO_COMMANDROUTER_AMQP_INSECUREPORTENABLED to true.

The server issues a warning on the console if HONO_COMMANDROUTER_AMQP_INSECUREPORT is set to the default secure AMQP port (5671).

The HONO_COMMANDROUTER_AMQP_INSECUREPORTBINDADDRESS variable can be used to specify the network interface that the port should be exposed on. By default the port is bound to the loopback device only, i.e. the port will only be accessible from the local host. This variable might be used to e.g. expose the non-TLS secured port on a local interface only, thus providing easy access from within the local network, while still requiring encrypted communication when accessed from the outside over public network infrastructure.

Setting this variable to 0.0.0.0 will let the port being bound to all network interfaces (be careful not to expose the port unintentionally to the outside world).

Dual Port

In test setups and some production scenarios Hono server may be configured to open one secure and one insecure port at the same time.

This is achieved by configuring both ports correctly (see above). The server will fail to start if both ports are configured to use the same port number.

Since the secure port may need different visibility in the network setup compared to the secure port, it has its own binding address HONO_COMMANDROUTER_AMQP_INSECUREPORTBINDADDRESS. This can be used to narrow the visibility of the insecure port to a local network e.g., while the secure port may be visible worldwide.

Ephemeral Ports

Both the secure as well as the insecure port numbers may be explicitly set to 0. The Command Router component will then use arbitrary (unused) port numbers determined by the operating system during startup.

Messaging Configuration

The Command Router component uses a connection to an AMQP 1.0 Messaging Network, an Apache Kafka cluster and/or Google Pub/Sub to

• receive command & control messages sent by downstream applications and to forward these commands on a specific address/topic so that they can be received by protocol adapters,
• send delivery failure command response messages in case no consumer exists for a received command (only with Kafka and Google Pub/Sub messaging),
• receive notification messages about changes to tenant/device/credentials data sent from the device registry.
• send an event message for Time until Disconnect Notification indicating the device readiness to receive commands.

Command messages are received on each configured messaging system.

For notification messages, the Kafka connection is used by default, if configured. Otherwise the AMQP messaging network or Google Pub/Sub is used.

AMQP 1.0 Messaging Network Connection Configuration

The connection to the AMQP 1.0 Messaging Network is configured according to the Hono Client Configuration with HONO_MESSAGING and HONO_COMMAND being used as ${PREFIX}. The properties for configuring response caching can be ignored.

Kafka based Messaging Configuration

The connection to an Apache Kafka cluster can be configured according to the Hono Kafka Client Configuration.

The following table provides an overview of the prefixes to be used to individually configure the Kafka clients used by the component. The individual client configuration is optional, a minimal configuration may only contain a common client configuration consisting of properties prefixed with HONO_KAFKA_COMMONCLIENTCONFIG_ and hono.kafka.commonClientConfig. respectively.

Google Pub/Sub Messaging Configuration

The connection to Google Pub/Sub is configured according to the Google Pub/Sub Messaging Configuration.

Tenant Service Connection Configuration

The Command Router component requires a connection to an implementation of Hono’s Tenant API in order to retrieve information for a tenant.

The connection to the Tenant Service is configured according to the Hono Client Configuration where the ${PREFIX} is set to HONO_TENANT and the additional values for response caching apply.

The adapter caches the responses from the service according to the cache directive included in the response. If the response doesn’t contain a cache directive no data will be cached.

Device Registration Service Connection Configuration

The Command Router component requires a connection to an implementation of Hono’s Device Registration API in order to retrieve registration status assertions for the target devices of incoming command messages.

The connection to the Device Registration Service is configured according to the Hono Client Configuration where the ${PREFIX} is set to HONO_REGISTRATION.

The adapter caches the responses from the service according to the cache directive included in the response. If the response doesn’t contain a cache directive no data will be cached.

Note that the adapter uses a single cache for all responses from the service regardless of the tenant identifier. Consequently, the Device Registration Service client configuration’s responseCacheMinSize and responseCacheMaxSize properties determine the overall number of responses that can be cached.

Data Grid Connection Configuration

The Command Router component uses a cache to store device connection information. Two implementations are provided, each built into its own Docker image. A deployment uses exactly one of them:

• the Infinispan based variant (hono-service-command-router-infinispan) which uses either an embedded cache or a remote data grid, accessed via the Infinispan Hotrod protocol, and
• the Redis based variant (hono-service-command-router-redis) which uses a remote Redis server.

Infinispan cache

The following table provides an overview of the configuration variables and corresponding command line options for configuring the common aspects of the Infinispan based variant:

The type of cache (embedded or remote) is determined during startup by means of the HONO_COMMANDROUTER_CACHE_REMOTE_SERVERLIST configuration variable. If the variable has a non empty value, a remote cache is configured. Otherwise, an embedded cache is configured.

Remote cache

The following table provides an overview of the configuration variables and corresponding system properties for configuring the connection to the data grid:

See also the Infinispan Hotrod client documentation.

Embedded cache

The following table provides an overview of the configuration variables and corresponding system properties for configuring the embedded cache:

Redis cache

The Redis based variant (hono-service-command-router-redis) stores device connection information in a remote Redis server. It uses the Quarkus Redis client to connect to the server, so you configure the connection with that client’s standard quarkus.redis properties.

The following table provides an overview of the most commonly used configuration variables and corresponding system properties for configuring the connection to the Redis server:

See the Quarkus Redis client reference for the full set of supported properties.

Connection pool sizing

The Redis client sends each Redis command over a connection from its pool. If all quarkus.redis.max-pool-size connections are busy, further Redis commands wait in a queue that holds up to quarkus.redis.max-pool-waiting entries. The client rejects the Redis commands that do not fit into the queue, and delivery of the affected device commands fails. The trace of such a failed delivery shows an io.vertx.core.http.ConnectionPoolTooBusyException with the message Connection pool reached max wait queue size of <n>. The redis_pool_rejected_total metric counts the rejected Redis commands. The redis_pool_ratio metric shows the share of the pool’s connections in use, and the redis_pool_queue_size metric shows the number of waiting Redis commands. If redis_pool_rejected_total increases, change the pool settings as follows:

• Raise quarkus.redis.max-pool-waiting first. Queueing usually adds only a few milliseconds to a Redis command.
• Raise quarkus.redis.max-pool-size if the queueing latency matters for command delivery.
• Keep the number of Command Router instances multiplied by quarkus.redis.max-pool-size below the connection limit of the Redis server (maxclients, which is 10000 by default) or of the managed Redis service.

Authentication Service Connection Configuration

The service requires a connection to an implementation of Hono’s Authentication API in order to authenticate and authorize client requests.

The connection is configured according to the Hono Client Configuration where the ${PREFIX} is set to HONO_AUTH. The properties for configuring the client’s response caching will be ignored because Hono’s Authentication Service does not allow caching of responses.

In addition to the standard client configuration properties, the following properties are supported:

Metrics Configuration

See Monitoring & Tracing Admin Guide for details on how to configure the reporting of metrics.