Backup and Restore

Backup and restore are administrative operations in Connectware. They are used to ensure reliable recovery of a Connectware installation after critical circumstances like hardware or software failures.

The required data to backup consist of:

1. Configuration files stored in shared docker container volumes. (This is less critical if all service commissioning files have been deployed using automated tools such as Ansible or similar, though.)

2. Postgresql database for user, role, and permission setup.

The recommended way of Connectware backup is to automate the backup procedure and execute it regularly.

To restore Connectware properly, the data stored in the Docker volumes should be consistent with the user and permissions data stored in the database. Any creation or update of a Connectware service may result in a database change. It is recommended to start a backup of the Docker volumes and the database backup at approximately the same time. Also, during the backup operation there should be no changes in the service configuration. Only in this case the consistency in the restored Connectware instance data can be guaranteed.

It is generally possible to restore the database across versions, too, i.e. from an older Connectware version’s backup into a newer Connectware version. However, as Connectware is enhanced and new features are added, the database structure might change sometimes between versions. If there is such a change in the database structure, additional steps will be required as described below. This situation is going to be explained in the status message of the restore operation.

Note

After a restore, Connectware should restart, and all Admin-UI browser sessions should be restarted as well.

Volumes

The most reliable way of creating a backup of all Connectware-related data is to backup all used Docker volumes completely. (This might not be possible due to unavailable root permissions, but is the most secure way of backup for all Connectware.) All volumes that have been defined from the docker-compose.yml file can be backed up, and if they are all restored on any docker environment, the Connectware installation is fully restored.

Service Commissioning File Volume

In order to back-up all service commissioning file data, the service-manager volume must be backed up.

(In versions up to 1.0.82, the protocol-mapper volume(s) were also containing service commissioning file data, but starting with 1.0.83, all this data is stored only within the service-manager volume and there is no longer any actively used protocol-mapper volume.)

Kubernetes

In a Kubernetes installation, the content of the persistent volume service-manager needs to be backed up. The name of the volume is starting with service-manager and may have certain suffixes. The way of creating a backup and restoring this depends on the storage providers that are used in a specific Kubernetes installation.

Docker-compose

In a docker-compose installation, the respective volume name is prepended by the docker-compose project name. This docker-compose project name is either given explicitly when running docker-compose e.g. using the -p option, or is taken from the current working directory, such as connectware. If the project name is connectware, the respective volume name is connectware_service-manager.

With the volume name given as e.g. connectware_service-manager, the actual storage location on a local harddisk can be looked up using the command docker volume inspect connectware_service-manager. In a default docker configuration, this is the local harddisk directory

• /var/lib/docker/volumes/connectware_service-manager/_data

This directory must be backed up regularly, and restored if a restore operation is needed.

An example command (requiring root privileges) which creates a compressed backup file /tmp/connectware-volume-backup.tgz looks like this:

cd  /var/lib/docker/volumes; \
tar cvfz /tmp/connectware-volume-backup.tgz \
./connectware_service-manager/_data

To restore the volume data from the created backup file, use the following command:

cd  /var/lib/docker/volumes; \
rm -rf ./connectware_service-manager/_data/*; \
tar xvfz /tmp/connectware-volume-backup.tgz

User Database

Connectware provides REST APIs to perform database maintenance operations for backup and restore.

For creating a backup, there is one REST call (shown below) to trigger the creation of the backup file. As this is a time-consuming operation, the creation of the backup file will be ongoing for some time and the status of the backup operation can be queried by another REST call. Once the creation of the backup file is completed, the resulting backup file can be downloaded by yet another REST call.

For restoring the state of a backup, there is one REST call where the backup file can be uploaded and also the restore operation can be started immediately.

Limitations of restore across versions

In a cross-version scenario, where a backup file from an older version is used for restoring in a newer version, some seldom version combinations can occur where the restore operation is being refused due to database structure changes.

In this case, Connectware database maintenance status message will report the last possible version that can be used to restore from the given backup file. If this backup file needs to be restored, it is necessary to downgrade the Connectware version to the reported older version.

If such a Connectware version downgrade is necessary, first ensure another backup of all data. Then, shut down Connectware and remove the volumes named postgresql and service-manager (of which a backup was stored beforehand). Then, follow the regular steps for the installation of the older version, as described in detail in the following article:

https://www.cybus.io/learn/installing-the-connectware

After the older version is installed, the backup file can be used for restore again. After restoring from the backup file, it is recommended to upgrade the Connectware version again, then creating a new backup file which is now from the newer version.

Example REST calls

To validate the status of a database maintenance operation, you can use the following CURL command:

curl -k --header "Content-Type: application/json" \
--request GET -u admin \
https://<HOST>/api/maintenance/db

The result looks like this:

{“running”:false,”backup”:{“succeded”:true,”id”:”1.0.73_2022-02-16T15:37:19.242Z”,”statusMessage”:”OK”,”startDate”:”2022-02-16T15:37:19.242Z”,”endDate”:”2022-02-16T15:37:19.270Z”},”restore”:{“succeded”:true,”id”:”1.0.73_2022-02-16T10:19:11.377Z”,”statusMessage”:”OK”,”startDate”:”2022-02-16T15:38:39.197Z”,”endDate”:”2022-02-16T15:38:39.385Z”}}

To start the creation of a backup file, call the following CURL command:

curl -k --request POST  -u admin \
https://<HOST>/api/maintenance/db/backup

On success, the request returns HTML status code 202 (Accepted).

To download the database backup in a file /tmp/connectware-db-backup.tgz, call the following CURL command:

curl -k --request GET  -u admin \
https://<HOST>/api/maintenance/db/backup \
--output /tmp/connectware-db-backup.tgz

To restore the database from the backup file /tmp/connectware-db-backup.tgz, call the following CURL command:

curl -k --request POST -u admin \
-F file=@/tmp/connectware-db-backup.tgz \
https://<HOST>/api/maintenance/db/restore

On success, the request returns HTML status code 202 (Accepted).

After each operation, the database maintenance operation status can be queried again (as shown in the first command).