Skip to content

Upgrading DRP and Components

DRP can upgrade itself. After upgrading DRP, content can be upgraded to the same version as the DRP minor version installed.

Pre-requisites

  • The dr-provision process must be able to locate its own binary.
    This is generally not a concern, but if something removed the running binary, then self-updating by any means will fail.

  • The dr-provision binary must be writable by the user that the process is running as, and the directory the binary is in must also be writable by that user.
    The final part of the DRP upgrade is to replace the current running executable with the new one and re-execute dr-provision. In order for that to happen, the user that the binary is running as must have write access to the binary and the directory that the binary is in.

  • DRP must be able to fix up its own capabilities.
    When running as a non-root user, DRP must be able to bind to some low ports to handle DHCP and TFTP traffic. On Linux systems, this is generally accomplished by using setcap on the binary to give it the additional capabilities it needs. For the self-upgrade process to be able to copy over the capabilities properly, it must be able to run sudo setcap with the same capability list the current binary has.

If any of these requirements are not met, then self-upgrade will not operate correctly, and manual steps will need to be taken to complete the upgrade.

Recommendations

  • Take complete backups of endpoints prior to starting any upgrades.

  • Make sure that the version of content matches the minor version of DRP that is running. It is safe for newer versions of DRP to run older content, but not the other way around. The CLI will return an error if the content is not compatible with the DRP version.

  • Upgrade DRP first, make sure everything is running successfully, then upgrade content from the catalog.

Upgrade DRP

Upgrading DRP can be done using the CLI, UX or API.

CLI

Start the upgrade process. 

$ drpcli system upgrade catalog:drp-stable

Note

To select a different version replace drp-stable with the drp-[version] of DRP. Examples include drp-stable, drp-tip, or drp-v4.14.21.

Monitor the status of the upgrade process. 

$ drpcli system upgrade status

Once the upgrade is complete, verify DRP has started successfully and is running the expected version. 

$ drpcli info get | jq -r '.version'
v4.14.21

UX

  • navigate to the Catalog menu item
  • find the Digital Rebar Provision entry
  • select the version you want to Upgrade (or downgrade) to in the v.Available (short for Available Versions) column
  • click on the green Install icon to the right, or the blue button top center that says Install 
        The archive has to be downloaded from the catalog, then uploaded to the endpoint, it may take some time.

API

The CLI is capable of determining the best upgrade path to use and combines the two-phase upgrade steps that are needed when using the API.

Stage the DRP Upgrade

Staging the upgrade involves uploading a dr-provision upgrade archive to the server, creating a rackn-system-upgrade-tracking profile to keep track of the upgrade progress, and (if in a consensus cluster) making sure that the upgrade archive and all of its components have replicated to all cluster members.

To stage an upgrade, you need an upgrade package. The easiest way to get one is to download it withdrpcli catalog item download drp to drp-upgrade.zip, although there are other options.

Once you have an upgrade package, you can stage the upgrade by issuing aPOST request to the API at /api/v3/system/upgrade/stage with the drp-upgrade.zip file contents as the body of the API request. If the request is accepted, you will get a StatusCreated response back, otherwise you will get an error back with information describing what is wrong.

Run the DRP Upgrade

Once an upgrade is staged, you can start the upgrade process by issuing a POST request to the API at /api/v3/system/upgrade/run with an empty request body. This will start the upgrade process in the background.

Monitor the DRP Upgrade

You can monitor the state of an upgrade using either GET /api/v3/profiles/rackn-system-upgrade-tracking or GET /api/v3/system/upgrade/status to get a point-in-time snapshot of the upgrade process. Both of these API requests will return the rackn-system-upgrade-tracking profile, which stores information in its Params about the ongoing state of the profile. The Params in the Profile are as follows:

status
A one-word description of the current status of the upgrade. Can be "staged" for a freshly-staged upgrade, "working" for an in-progress upgrade, "finished" when the upgrade is finished, and "failed" if the upgrade failed.
final-state
A one-line final summary of the upgrade process. It will only be present once the upgrade is finished or failed.
transcript
An array of one line status upgrades on the upgrade process. Can be used to get lower level information on how the upgrade is progressing.
working-node
The API URL of the node in the cluster that is currently being upgraded.
working-state
A one-line description of the current state of the upgrade on the working node.

Upgrade Content

Upgrading content or plugin providers can also be done using the CLI, UX or API.

CLI

Upgrading existing content can be done individually or collectively.

Upgrade specific content to the latest minor version. 

$ drpcli catalog item install univeral --version=v4.14

Note

The --version flag is optional. If not specified, the latest version of stable content will be installed. The install will verify prerequisites, if the prerequsites are not met, they will be updated to the latest minor version of the prerequisite as part of the install process.

Upgrade all installed content to the latest minor version of DRP. 

$ drpcli catalog item update-all --version=v4.14

Note

Only installed items that are in a catalog will be updated. Custom content or items that were installed directly without a catalog entry will not be updated.

UX

Upgrading Specific Content

  • Navigate to the Catalog menu item.
  • Select the version of the items you want to upgrade from the Available column.
  • Click on the green Install icon to the right, or the blue button top center that says Install.

Upgrading All Content

  • Navigate to the Catalog menu item.
  • Click on the blue button top center that says Upgrade.
  • Follow the prompts to upgrade all content including DRP.

Airgap

The steps for upgrading are similar for airgap, with the exception being that a catalog along with the content should be located somewhere accessible by the endpoints. This can be done by using a bundle that includes all these items.

Once the bundle has been made available locally on an endpoint, it is possible to upgrade DRP and place the catalog with contents in one step using the airgap-install.sh script.

Make sure DRP is stopped prior to running the install script. It will warn you if it is still running.

$ sudo systemctl stop dr-provision
$ ./airgap-install.sh upgrade
$ sudo systemctl start dr-provsion

Once the upgrade is complete, verify DRP has started successfully and is running the expected version. 

$ drpcli info get | jq -r '.version'
v4.14.21

Upgrading the content using the catalog flag to point to the local catalog. 

$ drpcli catalog item update-all --version=v4.14 --catalog file:///var/lib/dr-provision/tftpboot/files/rebar-catalog/rackn-catalog.json

Note

We point to the local filesystem in the example, but you can point to an endpoint using https://<ip-of-endpoint>:8090/files/rebar-catalog/rackn-catalog.json

If a more controlled approach is desired, or if you want to use a local static file server for the catalog and contents, it is possible to just explode the bundle to a location. 

$ ./airgap-install.sh explode

Then upgrade DRP and it's contents by pointing to the catalog that was exploded. 

$ drpcli catalog item update-all --version=v4.14 --catalog https://127.0.0.1:8090/files/rebar-catalog/rackn-catalog.json

Downgrade

Danger

Downgrading my cause loss of settings or data.

The procedures for downgrading are intended soley for standalone endpoints. Contact support for help downgrading clustered systems.

Downgrading DRP from one minor release to another requires additional steps as the underlying database that backs the service may change between minor point releases.

Warning

Always perform these downgrade steps if you are moving from one higher point release to a lower point release (eg v4.6.x to v4.5.x).

All downgrade steps and examples below are run at the shell of the server where the dr-provision service is installed and running, as the root user, unless installed as a non-privileged user.

Backup DRP First

The system should be backed up before upgrading.

Stop DRP

The dr-provision service needs to be stopped for a downgrade procedure in order to convert the database records to flat JSON text files. This is called "humanize", as it turns the database records in to human-readable components.

$ sudo systemctl stop dr-provision

Humanize the Database

The dr-provision binary has a special flag --humanize which converts the current database format components in to human readable JSON text files. You must run the same dr-provision version binary as the database format is using. In addition, if you have installed DRP in a location other than the default production install path (/var/lib/dr-provision), you must also specify where the DRP base directory is with the --base-root flag.

Once the dr-provision service is stopped, run the humanize step.

$ DRP_ROOT="/var/lib/dr-provision"
$ dr-provision --humanize --base-root=$DRP_ROOT

To verify that the humanize step completed properly, look at the base directory for a new directory named digitalrebar.

Install DRP

To install the older DRP binary, manually extract the binary out of the distributed file (even though the file ends in .zip, it is a gzipped tarball. It can be downloaded using the catalog. 

$ drpcli catalog item download drp --version=v4.13.6

Explode the archive file. 

$ tar xzvf drp.zip

Copy the older DRP binary in place: 

$ mv /usr/local/bin/dr-provision /usr/local/bin/dr-provision.old
$ cp bin/linux/amd64/dr-provision /usr/local/bin/

Start the dr-provision Service

$ sudo systemctl start dr-provision

Note

The startup process may take some time (up to 15 minutes), depending on the number of objects that need to be converted.

Verify the service is running the expected version via the command line tool: 

$ drpcli info get | jq -r '.version'