File Backup

Customize the files backup and restore to and from a backend storage.

1: File backup configuration
2: File Backup API

1 - File backup configuration

Customize the files backup and restore to and from a backend storage.

Properties

To control all aspects of the file backup behavior.

Property	Type	Default	Description
featureId	string	BackupAndRestore	Feature unique identifier in the scope of the edge digital twin
type	string	file	Type of the files that are backed up by this feature
context	string	edge	Context of the files backed up by this feature, unique in the scope of the `type`
dir	string		Directory to be backed up
mode	string	strict	Restriction on directories that can be dynamically selected for a backup, the supported modes are: strict, lax and scoped
backupCmd	string		Command to be executed before the backup is done
restoreCmd	string		Command to be executed after the restore
singleUpload	bool	false	Forbid triggering of new backups when there is a backup in progress
checksum	bool	false	Send MD5 checksum for backed up files to ensure data integrity
stopTimeout	string	30s	Time to wait for running backups to finish as a sequence of decimal numbers, each with optional fraction and a unit suffix, such as: 300ms, 1.5h, 10m30s, etc., time units are: ns, us (or µs), ms, s, m, h
keepUploaded	bool	false	Keep successfully uploaded backups locally
storage	string	./storage	Directory where backups and downloads will be stored
Upload/Download - TLS
serverCert	string		PEM encoded certificate file for secure uploads and downloads
Auto backup
active	bool	false	Activate periodic backups
activeFrom	string		Time from which periodic backups should be active, in RFC 3339 format, if omitted (and `active` flag is set) current time will be used as start of the periodic backups
activeTill	string		Time till which periodic backups should be active, in RFC 3339 format, if omitted (and `active` flag is set) periodic backups will be active indefinitely
period	string	10h	Backup period as a sequence of decimal numbers, each with optional fraction and a unit suffix, such as: 300ms, 1.5h, 10m30s, etc., time units are: ns, us (or µs), ms, s, m, h
Local connectivity
broker	string	tcp://localhost:1883	Address of the MQTT server/broker that the file backup will connect for the local communication, the format is: `scheme://host:port`
username	string		Username that is a part of the credentials
password	string		Password that is a part of the credentials
Local connectivity - TLS
caCert	string		PEM encoded CA certificates file
cert	string		PEM encoded certificate file to authenticate to the MQTT server/broker
key	string		PEM encoded unencrypted private key file to authenticate to the MQTT server/broker
Logging
logFile	string	log/file-backup.log	Path to the file where log messages are written
logLevel	string	INFO	All log messages at this or a higher level will be logged, the log levels in descending order are: ERROR, WARN, INFO, DEBUG and TRACE
logFileCount	int	5	Log file maximum rotations count
logFileMaxAge	int	28	Log file rotations maximum age in days, use 0 to not remove old log files
logFileSize	int	2	Log file size in MB before it gets rotated

Example

The minimal required configuration that enables backing up a directory and sets the file type to config.

{
    "type": "config",
    "dir": "/var/tmp/file-backup",
    "mode": "scoped",
    "storage": "/var/lib/file-backup",
    "logFile": "/var/log/file-backup/file-backup.log"
}

Template

The configuration can be further adjusted according to the use case. The following template illustrates all possible properties with their default values.

Be aware that some combinations may be incompatible.

{
  "featureId": "BackupAndRestore",
  "type": "file",
  "context": "edge",
  "dir": "",
  "mode": "strict",
  "backupCmd": "",
  "restoreCmd": "",
  "singleUpload": false,
  "checksum": false,
  "stopTimeout": "30s",
  "keepUploaded": false,
  "storage": "./storage",
  "serverCert": "",
  "active": false,
  "activeFrom": "",
  "activeTill": "",
  "period": "10h",
  "broker": "tcp://localhost:1883",
  "username": "",
  "password": "",
  "caCert": "",
  "cert": "",
  "key": "",
  "logFile": "log/file-backup.log",
  "logLevel": "INFO",
  "logFileCount": 5,
  "logFileMaxAge": 28,
  "logFileSize": 2
}

2 - File Backup API

The file backup service allows you to backup and restore data to and from a backend storage.

Backup

Create a file backup.

Request

Hono Command: command//<name>:<namespace>/req//backup

Ditto Message:

Name Value Description
topic <name>/<namespace>/things/live/messages/backup Information about the affected Thing and the type of operation
path /features/BackupAndRestore/inbox/messages/backup A path to the BackupAndRestore Feature, it’s message channel, and backup command
Headers Additional headers
response-required true/false If response is required
content-type application/json The content type
correlation-id UUID Used for correlating protocol messages, the same correlation-id as the sent back response message
Value
correlationID UUID Identifier of the backup file
providers The providers of the restore command
options
backup.dir A local directory, to be backed up
https.url The URL for restoring the backed up directory

Name	Value	Description
topic	`<name>/<namespace>/things/live/messages/backup`	Information about the affected Thing and the type of operation
path	`/features/BackupAndRestore/inbox/messages/backup`	A path to the `BackupAndRestore` Feature, it’s message channel, and `backup` command
Headers		Additional headers
response-required	true/false	If response is required
content-type	`application/json`	The content type
correlation-id	UUID	Used for correlating protocol messages, the same correlation-id as the sent back response message
Value
correlationID	UUID	Identifier of the backup file
providers		The providers of the restore command
options
backup.dir		A local directory, to be backed up
https.url		The URL for restoring the backed up directory

Example : Back up a directory.

Topic: command//edge:device/req//backup

{
	"topic":"edge/device/things/live/messages/backup",
	"headers":{
		"response-required":true,
		"content-type":"application/json",
		"correlation-id":"<UUID>"
	},
	"path":"/features/BackupAndRestore/inbox/messages/backup",
	"value":{
		"correlationID":"upload-id-1704439450#n",
		"providers":{},
		"options":{
			"backup.dir":"/var/tmp/backup",
			"https.url":""
		}
	}
}

Response

Hono Command : command//<name>:<namespace>/res//backup

Ditto Message:

Name Value Description
topic <name>/<namespace>/things/live/messages/backup Information about the affected Thing and the type of operation
path /features/BackupAndRestore/outbox/messages/backup A path to the BackupAndRestore Feature, it’s message channel, and backup command
Headers Additional headers
content-type application/json The content type
correlation-id <UUID> The same correlation id as the request message
Status Status of the backup operation

Name	Value	Description
topic	`<name>/<namespace>/things/live/messages/backup`	Information about the affected Thing and the type of operation
path	`/features/BackupAndRestore/outbox/messages/backup`	A path to the `BackupAndRestore` Feature, it’s message channel, and `backup` command
Headers		Additional headers
content-type	`application/json`	The content type
correlation-id	<UUID>	The same correlation id as the request message
Status		Status of the `backup` operation

Example : Successful response of a backup operation.

Topic: `command//edge:device/res//backup``

{
	"topic":"edge/device/things/live/messages/backup",
	"headers":{
		"content-type":"application/json",
		"correlation-id":"<UUID>"
	},
	"path":"/features/BackupAndRestore/outbox/messages/backup",
	"status":204
}

Restore

Restore the backed up files or directory from a backend service.

Request

Hono Command: command//<name>:<namespace>/req//restore

Ditto Message:

Name Value Description
topic <name>/<namespace>/things/live/messages/restore Information about the affected Thing and the type of operation
path /features/BackupAndRestore/inbox/messages/restore A path to the BackupAndRestore Feature, it’s message channel, and restore command
Headers Additional headers
response-required true/false If response is required
content-type application/json The content type
correlation-id UUID Used for correlating protocol messages, the same correlation-id as the sent back response message
Value
correlationID other UUID Identifier of the restored file
providers Storage provider, one of aws, azure, generic
options Options are specific for each provider
backup.dir A local directory, which to be backed up and then uploaded, using a storage provider of choice and temporary credentials
https.url The URL for restoring the backed up directory

Name	Value	Description
topic	`<name>/<namespace>/things/live/messages/restore`	Information about the affected Thing and the type of operation
path	`/features/BackupAndRestore/inbox/messages/restore`	A path to the `BackupAndRestore` Feature, it’s message channel, and `restore` command
Headers		Additional headers
response-required	true/false	If response is required
content-type	`application/json`	The content type
correlation-id	UUID	Used for correlating protocol messages, the same correlation-id as the sent back response message
Value
correlationID	other UUID	Identifier of the restored file
providers		Storage provider, one of `aws`, `azure`, `generic`
options		Options are specific for each provider
backup.dir		A local directory, which to be backed up and then uploaded, using a storage provider of choice and temporary credentials
https.url		The URL for restoring the backed up directory

Example : Restore a backup from a storage provider.

Topic: command//edge:device/req//restore

{
	"topic":"edge/device/things/live/messages/restore",
	"headers":{
		"response-required":true,
		"content-type":"application/json",
		"correlation-id":"<UUID>"
	},
	"path":"/features/BackupAndRestore/inbox/messages/restore",
	"value":{
		"correlationID":"upload-id-1704439450#n",
		"providers":{},
		"options":{
			"backup.dir":"/var/tmp/backup",
			"https.url":"https://raw.githubusercontent.com/eclipse-kanto/container-management/main/containerm/pkg/testutil/config/"
		}
	}
}

Response

Hono Command : command//<name>:<namespace>/res//restore

Ditto Message:

Name Value Description
topic <name>/<namespace>/things/live/messages/restore Information about the affected Thing and the type of operation
path /features/BackupAndRestore/outbox/messages/restore A path to the BackupAndRestore Feature, it’s message channel, and restore command
Headers Additional headers
content-type application/json The content type
correlation-id <UUID> The same correlation id as the request message
Status Status of the operation restore

Name	Value	Description
topic	`<name>/<namespace>/things/live/messages/restore`	Information about the affected Thing and the type of operation
path	`/features/BackupAndRestore/outbox/messages/restore`	A path to the `BackupAndRestore` Feature, it’s message channel, and `restore` command
Headers		Additional headers
content-type	`application/json`	The content type
correlation-id	<UUID>	The same correlation id as the request message
Status		Status of the operation restore

Example : Response of a successful restore operation.

Topic: `command//edge:device/res//restore``

{
	"topic":"edge/device/things/live/messages/restore",
	"headers":{
		"content-type":"application/json",
		"correlation-id":"<UUID>"
	},
	"path":"/features/BackupAndRestore/outbox/messages/restore",
	"status": 204
}