Upgrading the orchestrator

Upgrading the orchestrator can be done either in-place or by setting up a new orchestrator next to the old one and migrating the state from the old to the new instance. The sections below describe the upgrade procedure for each of both situations. These procedures can be used for major and non-major version upgrades.

Note

Make sure to read the new version’s changelog for any version specific upgrade notes, before proceeding with any of the upgrade procedures mentioned below.

Upgrading the orchestrator in-place

This section describes how to upgrade an orchestrator in-place.

Note

Pre-requisite

  • Before upgrading the orchestrator to a new major version, make sure the old orchestrator is at the latest version available within its major.

  • Upgrades should be done one major version at a time. Upgrading from major version X to major version X+2, should be done by upgrading from X to X+1 and then from X+1 to X+2.

  1. Halt all environments (by pressing the STOP button in the web-console for each environment).

  2. Create a backup of the database:

    pg_dump -U <db_user> -W -h <host> <db_name> > <db_dump_file>

    # Write the dump into the filesystem of the container
podman exec -it inmanta-db pg_dump -U <db_user> -W -h localhost -f /tmp/inmanta.sql <db_name>
# Copy the dump to the filesystem of the host
podman cp inmanta-db:/tmp/inmanta.sql .

  3. Update the artifact repository to the repository for the new major release:

    If you are doing an upgrade to a new major version, replace the content of the /etc/yum.repos.d/inmanta.repo file with the content for the new ISO version. This information can be obtained from the installation documentation page for the new ISO version.

    Replace the image tag in /home/inmanta/.config/containers/systemd/inmanta-server.image with the one for the new ISO version. This information can be obtained from the installation documentation page for the new ISO version.

  4. Upgrade the Inmanta server by running the following command(s). It might take some time before the orchestrator goes up, as some database migrations will be done.

    sudo dnf update inmanta-service-orchestrator-server

    systemctl --user daemon-reload
systemctl --user restart inmanta-server-image.service
systemctl --user restart inmanta-server.service

  5. When accessing the web console, all the environments will be visible, and still halted.

  6. One environment at a time:

    a. In the Desired State page of the environment, click Update project & recompile, accessible via the dropdown of the Recompile button. (/console/desiredstate?env=<your-env-id>).

    1. Resume the environment by pressing the green Resume button in the bottom left corner of the console.

    Warning

    Make sure the compilation has finished and was successful.

Upgrading by migrating from one orchestrator to another orchestrator

This document describes how to upgrade to a new version of the orchestrator by setting up a new orchestrator next to the existing orchestrator and migrating all the state from the existing to the new orchestrator. This procedure should be followed when an in-place upgrade of the orchestrator is not possible e.g. when the operating system needs to be upgraded alongside the orchestrator.

Terminology

The procedure below describes how to migrate from one running orchestrator denoted as the ‘old orchestrator’ to another one denoted as the ‘new orchestrator’.

Procedure

Note

Pre-requisite

  • Before upgrading the orchestrator to a new major version, make sure the old orchestrator is at the latest version available within its major.

  • Upgrades should be done one major version at a time. Upgrading from major version X to major version X+2, should be done by upgrading from X to X+1 and then from X+1 to X+2.

  1. [New Orchestrator]: Make sure the desired version of the orchestrator is installed, by following the installation instructions (see RPM installation or container-based installation) and set up a project to validate that the orchestrator is configured correctly (config, credentials, access to packages, etc.).

  2. [Old Orchestrator] Halt all environments (by pressing the STOP button in the web-console for each environment).

  3. [Old Orchestrator] Stop and disable the server:

    sudo systemctl disable --now inmanta-server.service

    Make sure that the following section is not part of the /home/inmanta/.config/containers/systemd/inmanta-server.container file to prevent it from starting at boot.

    [Install]
WantedBy=default.target

    Activate the new config and stop the server.

    systemctl --user daemon-reload
systemctl --user stop inmanta-server.service

  4. [Old Orchestrator] Make a dump of the server database using pg_dump.

    pg_dump -U <db_user> -W -h <host> -f <db_dump_file> <db_name>

    # Write the dump into the filesystem of the container
podman exec -it inmanta-db pg_dump -U <db_user> -W -h localhost -f /tmp/inmanta.sql <db_name>
# Copy the dump to the filesystem of the host
podman cp inmanta-db:/tmp/inmanta.sql .

  5. [New Orchestrator] Make sure the server is stopped:

    sudo systemctl stop inmanta-server.service

    systemctl --user stop inmanta-server.service

  6. [New Orchestrator] If the database already exists, drop and recreate it. It’s not possible to remove a database that is in use. The command below assumes that the postgres database exists and you want to delete and recreate a database <db_name> which has a different name than postgres.

    # drop the database
psql -h <host> -U <db_user> -W -c "drop database <db_name>;" postgres

# re-create it
sudo -u postgres -i bash -c "createdb -O <db_user> <db_name>"

    # drop the database
podman exec -it inmanta-db psql -U <db_user> -W -h localhost -c "drop database <db_name>;" postgres

# re-create it
podman exec -it inmanta-db bash -c "createdb -U <db_user> -h localhost -W -O <db_user> <db_name>"

  7. [New Orchestrator] Load the dump of the server database using psql.

    psql -U <db_user> -W -h <host> -f <db_dump_file> <db_name>

    podman cp <db_dump_file> inmanta-db:/tmp
podman exec -it inmanta-db psql -U <db_user> -W -h localhost -f /tmp/<db_dump_file> <db_name>

  8. [New Orchestrator] Start the orchestrator service, it might take some time before the orchestrator goes up, as some database migration will be done:

    sudo systemctl enable --now inmanta-server.service

    Make sure that the following section is part of the /home/inmanta/.config/containers/systemd/inmanta-server.container file to make the server start at boot.

    [Install]
WantedBy=default.target

    Activate the new config and start the server.

    systemctl --user daemon-reload
systemctl --user start inmanta-server.service

  9. [New Orchestrator] When accessing the web console, all the environments will be visible, and still halted.

  10. [New Orchestrator] One environment at a time:

    a. In the Desired State page of the environment, click Update project & recompile, accessible via the dropdown of the Recompile button. (/console/desiredstate?env=<your-env-id>).

    1. Resume the environment by pressing the green Resume button in the bottom left corner of the console.

    Warning

    Make sure the compilation has finished and was successful.