Image Builder

The image builder content pack allows an operator to create a workflow to build a gold master system and to capture a filesystem tarball image of that system as a single artifact image. The image is suitable for deployment to new systems via the image-deploy plugin.

In addition, the tools used in this content pack to capture an image of a workflow built image should also be sufficient to capture so called brown field images. This would allow an operator to essentially "cut-n-paste" copies of existing servers in their operational environment.

Currently, the Image Builder content only supports Linux type systems. If you wish to build Microsoft Windows images, please consider using a different tool like Hashicorp's Packer.

The image-builder content pack currently is only tested with CentOS and Ubuntu based Linux distros. However, most any Linux distribution should be supportable with very minor changes to the content.

The Process¶

The basic process that this content pack employs is as follows:

create a workflow to build a new machine, often referred to as a "gold image", from scratch
use stages to customize the contents/configuration of your image
capture the image as a filesystem tarball, and compress it
stage the image to the DRP Endpoint in the files space for deployment

The process is relatively opinionated in some aspects (repo management, installation of cloud-init for image deployment customizations, etc.).

It is entirely likely that you may need to customize either the Stages or specific tasks attached to the Stages to meet your use case requirements.

Customizing Your Image - Uploaded Image Name¶

The default image name that is generated may or may not suit your use case requirement. If the image name (and accompanying Manifest file) need to carry a custom formatted name when uploaded to the DRP Endpoint, you can do so by use of the following three Params:

image-builder/upload-path
image-builder/upload-rootfs-name
image-builder/upload-manifest-name

Please review the documentation from each of those Params for more complete details.

Customizing Your Image - Image Contents Changes¶

The current image builder utilizes the OS installers (eg Anaconda for RedHat/CentOS, Debian Installer for Debian/Ubuntu, etc...). During the installation, the Digital Rebar agent (runner) is started in the Post section of these installers. Workflow Stages/Tasks can be run to customize the OS image.

The image-builder workflows each contain two Flexiflow Stages that will allow the operator to dynamically inject Tasks in to the workflow. These tasks have access to the final installed filesystem via the chroot environment. Tasks can install, customize, or alter the final image for your needs.

The Tasks can be added prior to the image-builder-linux Stage which performs package/repo management functions, system update, etc. and after the image-builder-linux Stage, but before the image-builder-capture Stage. The first flexiflow injection lets the operator add customizations to the base OS, prior to administrative and cleanup tasks, while the second, allows for post-cleanup tasks for any additional final cleanup, or Image personalization customizations desired.

To utilize the Flexiflow stages, your target machine you are building your images on must have the Parameter(s) image-builder/pre-builder-flexiflow and/or the image-builder/post-builder-flexiflow on it. Each is an array list of strings that should reference Task name(s) to add, in order.

These tasks will be injected dynamically in to the Workflow to allow customization, at the Pre or Post stage - depending on which Param values you set.

An example of adding repositories that are not part of the base installer definitions, installing packages, and then subsequently removing the repository definitioin might look something like the following:

image-builder/pre-builder-flexiflow:
  - my-repository-add
  - my-install-my-packages
  - my-repository-remove
image-builder/post-builder-flexiflow:
  - my-add-etc-issue

In our theoretical example above, we have 3 tasks that will be injected in the Pre stage. The first should add a new repository to the installed system (for example EPEL). The second would install some packages that are contained in the example repository. The third then removes the example repository from the sysmem - so the final installed system is in a clean state without those repos.

After the image-builder-linux Stage/Tasks run, then the Post tasks will be injected. In this case my-add-etc-issue. In this hypothetical task, the operator could add a signature to the end of /etc/issue to identify the image build uniquely to your organization.

Note

The above tasks are examples, and do not actually exist, the operator would have to create them as a part of a content pack or individual objects added on to the DRP Endpoint.

Tutorial Videos¶

The following tutorial videos have been made that outline and show the use of this content pack in conjunction with the image-deploy plugin.

Red Hat Enterprise Linux 7.7 (RHEL 7.7)
Digital Rebar: Immutable Image Deployment Demo (CentOS)

Image Requirements¶

If images built by the image-builder content pack are going to be deployed via the image-deploy plugin capabilities, you will need to observe the following requirements for image-deploy to successfully deploy the image.

There must be a base /curtin directory in the image filesystem - this is used by the deployment tooling to inject pieces, handle adding DRP Agent, and other tasks.
(optional): install of cloud-init which is used for initial bootstrap configuration of an image-deploy system

Image Build versus Image Deploy Requirements¶

One common pitfall of image build and deploy solutions, is ensuring your source image you build supports the target platforms you deploy to. This is evident in two primary areas of concern:

Legacy BIOS versus UEFI BIOS boot modes
Hardware drivers

You must verify that the image build platform and OS deployment solution encompasses the appropriate BIOS modes and hardware drivers for your target deployment systems.

Legacy BIOS versus UEFI BIOS boot modes¶

For Linux systems, specific packages may need to be included to support UEFI boot mode, when the original image builder (eg gold master) system is in Legacy BIOS mode. How to do this is dependent on the specific Linux distro family you are deploying. An example for CentOS (and likely other Redhat derived OS types), is to ensure the gold image system includes the grub2-efi-x64-modules package, even though the system may be in Legacy BIOS mode (as it won't be installed by default). More details related to this example can be found in the Operations documentation as Image Deploy.

Other Linux distros may require that you build a UEFI based image for UEFI system deployment; a Legacy BIOS system may not support adding UEFI packages. This appears to be the case with Ubuntu based distributions.

Hardware Drivers¶

Similar to the Legacy -vs- UEFI boot modes, you must ensure you inject appropriate hardware drivers for your target platform(s) that you will be deploying to, which may not match your gold master image builder platform you prepare your image on.

Some examples may include NIC drivers, Storage drivers, etc.

Uploading Images¶

At the conclusion of the image-capture task (in the image-builder-linux stage), the image will be uploaded to the HTTP fileserver space on the DRP Endpoint. To successfully upload the image, the image builder content uses the ExtraClaims to add additional scope access to the files API service. This is used to allow the use of drpcli files upload .... command to store the image on the DRP Endpoint for use by the image-deploy plugin.

Note

Older versions (prior to v4.3.0) used two Params to define the Username and Password with access to upload images to your DRP Endpoint. The use of unsafe/rs-username and unsafe/rs-password Params have been removed.

Using the Image¶

Once the image build process has been completed, you will need to locate the HTTP file server URL location for use in the image deploy image-deploy/image-url Param. To find the uploaded URL, review the output of the image-capture Job Log. You should find a output similar to:

| Image Builder created image information:
| --------------------------------------------------------------------------------
|         filename:  /RS_IMAGE/tarball.tgz
|            ident:  cf99cd80eb
|             date:  Tue Aug 27 18:44:35 UTC 2019
|             size:  517036600
|           md5sum:  1f40ac1bac2eb3ea99cebc38b807c15f
|   upload tarball:  images/rhel-tarball-20190827-184435-cf99cd80eb.tgz
|  upload manifest:  images/rhel-MANIFEST-20190827-184435-cf99cd80eb.txt

The relevant piece is the images/rhel-tarball-20190827-184435-cf99cd80eb.tgz. Using this example, you can set the image-deploy/image-url to:

{{ .ProvisionerURL }}/files/images/rhel-tarball-20190827-184435-cf99cd80eb.tgz

Example Image Deploy Profile¶

Below is an example Profile that could be used to reference an Image Builder built image for deployment by the image-deploy plugin. Note that these values are examples and will be unique to your built image (YAML format).

---
Name: "my-image-deploy"
Meta:
  color: "purple"
  icon: "wizard"
  title: "Built by Image Builder"
Params:
  image-deploy/image-os: "linux"
  image-deploy/image-type: "tgz"
  image-deploy/image-url: "http://192.168.124.1:8091/files/images/rhel-tarball-20190827-184435-cf99cd80eb.tgz"
  kexec-ok: true

Note that the use of kexec-ok: true is not required, but often helpful in enabling extremely rapid image deployment capabilities. It is also heavily dependent on the built source image containing kexec support in the compiled in kernel.

Prerequisites¶

The image-builder content pack has Prerequisites necessary to operate correctly. In addition to any content packs that define/describe BootEnvs and necessary workflow stages (for example RHEL 7.7 requiers os-other).

Please ensure you have the appropriate content packs to support workflows for building images as desired. The Preqrequisistes system does not call out content or plugins that are optional for building your image.