22.26. image-builder - Image Builder¶
The following documentation is for Image Builder (image-builder) content package at version v4.10.0-alpha00.46+g1471cf9bbea9d50f3360a078e87e85f65fff4c7c.
22.26.1. Overview¶
The image builder content pack allows an operator to create a workflow
to build a gold master system and to capture a filesystem tarball image
of that system as a single artifact image. The image is suitable for
deployment to new systems via the image-deploy
plugin.
In addition, the tools used in this content pack to capture an image of a workflow built image should also be sufficient to capture so called brown field images. This would allow an operator to essentially “cut-n-paste” copies of existing servers in their operational environment.
Currently, the Image Builder content only supports Linux type systems. If you wish to build Microsoft Windows images, please consider using a different tool like Hashicorp’s Packer.
The image-builder
content pack currently is only tested with CentOS and
Ubuntu based Linux distros. However, most any Linux distribution should be
supportable with very minor changes to the content.
22.26.1.1. The Process¶
The basic process that this content pack employs is as follows:
create a workflow to build a new machine, often referred to as a “gold image”, from scratch
use stages to customize the contents/configuration of your image
capture the image as a filesystem tarball, and compress it
stage the image to the DRP Endpoint in the
files
space for deployment
The process is relatively opinionated in some aspects (repo management, installation
of cloud-init
for image deployment customizations, etc.).
It is entirely likely that you may need to customize either the Stages or specific tasks attached to the Stages to meet your use case requirements.
22.26.1.2. Customizing Your Image - Uploaded Image Name¶
The default image name that is generated may or may not suit your use case requirement. If the image name (and accompanying Manifest file) need to carry a custom formatted name when uploaded to the DRP Endpoint, you can do so by use of the following three Params:
image-builder/upload-path
image-builder/upload-rootfs-name
image-builder/upload-manifest-name
Please review the documentation from each of those Params for more complete details.
22.26.1.3. Customizing Your Image - Image Contents Changes¶
The current image builder utilizes the OS installers (eg Anaconda for RedHat/CentOS, Debian Installer for Debian/Ubuntu, etc…). During the installation, the Digital Rebar agent (runner) is started in the Post section of these installers. Workflow Stages/Tasks can be run to customize the OS image.
The image-builder
workflows each contain two Flexiflow Stages that will allow the
operator to dynamically inject Tasks in to the workflow. These tasks have access
to the final installed filesystem via the chroot environment. Tasks can install,
customize, or alter the final image for your needs.
The Tasks can be added prior to the image-builder-linux
Stage which performs
package/repo management functions, system update, etc. and after the image-builder-linux
Stage, but before the image-builder-capture
Stage. The first flexiflow injection
lets the operator add customizations to the base OS, prior to administrative and
cleanup tasks, while the second, allows for post-cleanup tasks for any additional
final cleanup, or Image personalization customizations desired.
To utilize the Flexiflow stages, your target machine you are building your images on
must have the Parameter(s) image-builder/pre-builder-flexiflow
and/or the
image-builder/post-builder-flexiflow
on it. Each is an array list of strings that
should reference Task name(s) to add, in order.
These tasks will be injected dynamically in to the Workflow to allow customization, at the Pre or Post stage - depending on which Param values you set.
An example of adding repositories that are not part of the base installer definitions, installing packages, and then subsequently removing the repository definitioin might look something like the following:
image-builder/pre-builder-flexiflow: - my-repository-add - my-install-my-packages - my-repository-remove image-builder/post-builder-flexiflow: - my-add-etc-issue
In our theoretical example above, we have 3 tasks that will be injected in the Pre stage. The first should add a new repository to the installed system (for example EPEL). The second would install some packages that are contained in the example repository. The third then removes the example repository from the sysmem - so the final installed system is in a clean state without those repos.
After the image-builder-linux
Stage/Tasks run, then the Post tasks will
be injected. In this case my-add-etc-issue
. In this hypothetical task,
the operator could add a signature to the end of /etc/issue
to identify
the image build uniquely to your organization.
Note
The above tasks are examples, and do not actually exist, the operator would have to create them as a part of a content pack or individual objects added on to the DRP Endpoint.
22.26.1.4. Tutorial Videos¶
The following tutorial videos have been made that outline and show the use of this
content pack in conjunction with the image-deploy
plugin.
Red Hat Enterprise Linux 7.7 (RHEL 7.7)
22.26.1.5. Image Requirements¶
If images built by the image-builder
content pack are going to be deployed via the
image-deploy
plugin capabilities, you will need to observe the following requirements
for image-deploy
to successfully deploy the image.
There must be a base
/curtin
directory in the image filesystem - this is used by the deployment tooling to inject pieces, handle adding DRP Agent, and other tasks.(optional): install of
cloud-init
which is used for initial bootstrap configuration of animage-deploy
system
22.26.1.6. Image Build versus Image Deploy Requirements¶
One common pitfall of image build and deploy solutions, is ensuring your source image you build supports the target platforms you deploy to. This is evident in two primary areas of concern:
Legacy BIOS versus UEFI BIOS boot modes
Hardware drivers
You must verify that the image build platform and OS deployment solution encompasses the appropriate BIOS modes and hardware drivers for your target deployment systems.
22.26.1.6.1. Legacy BIOS versus UEFI BIOS boot modes¶
For Linux systems, specific packages may need to be included to support UEFI boot mode,
when the original image builder (eg gold master) system is in Legacy BIOS mode. How
to do this is dependent on the specific Linux distro family you are deploying. An
example for CentOS (and likely other Redhat derived OS types), is to ensure the
gold image system includes the grub2-efi-x64-modules
package, even though the
system may be in Legacy BIOS mode (as it won’t be installed by default). More details
related to this example can be found in the Operations documentation as Building And Deploying CentOS Images.
Other Linux distros may require that you build a UEFI based image for UEFI system deployment; a Legacy BIOS system may not support adding UEFI packages. This appears to be the case with Ubuntu based distributions.
22.26.1.6.2. Hardware Drivers¶
Similar to the Legacy -vs- UEFI boot modes, you must ensure you inject appropriate hardware drivers for your target platform(s) that you will be deploying to, which may not match your gold master image builder platform you prepare your image on.
Some examples may include NIC drivers, Storage drivers, etc.
22.26.1.7. Uploading Images¶
At the conclusion of the image-capture
task (in the image-builder-linux
stage),
the image will be uploaded to the HTTP fileserver space on the DRP Endpoint. To successfully
upload the image, the image builder content uses the ExtraClaims
to add additional scope
access to the files
API service. This is used to allow the use of drpcli files upload ....
command to store the image on the DRP Endpoint for use by the image-deploy
plugin.
Note
Older versions (prior to v4.3.0) used two Params to define the Username and Password
with access to upload images to your DRP Endpoint. The use of unsafe/rs-username
and unsafe/rs-password
Params have been removed.
22.26.1.8. Using the Image¶
Once the image build process has been completed, you will need to locate the HTTP file server
URL location for use in the image deploy image-deploy/image-url
Param. To find the uploaded
URL, review the output of the image-capture
Job Log. You should find a output similar to:
| Image Builder created image information: | -------------------------------------------------------------------------------- | filename: /RS_IMAGE/tarball.tgz | ident: cf99cd80eb | date: Tue Aug 27 18:44:35 UTC 2019 | size: 517036600 | md5sum: 1f40ac1bac2eb3ea99cebc38b807c15f | upload tarball: images/rhel-tarball-20190827-184435-cf99cd80eb.tgz | upload manifest: images/rhel-MANIFEST-20190827-184435-cf99cd80eb.txt
The relevant piece is the images/rhel-tarball-20190827-184435-cf99cd80eb.tgz
. Using this
example, you can set the image-deploy/image-url
to:
{{ .ProvisionerURL }}/files/images/rhel-tarball-20190827-184435-cf99cd80eb.tgz
22.26.1.8.1. Example Image Deploy Profile¶
Below is an example Profile that could be used to reference an Image Builder built image
for deployment by the image-deploy
plugin. Note that these values are examples and
will be unique to your built image (YAML format).
--- Name: "my-image-deploy" Meta: color: "purple" icon: "wizard" title: "Built by Image Builder" Params: image-deploy/image-os: "linux" image-deploy/image-type: "tgz" image-deploy/image-url: "http://192.168.124.1:8091/files/images/rhel-tarball-20190827-184435-cf99cd80eb.tgz" kexec-ok: true
Note that the use of kexec-ok: true
is not required, but often helpful in enabling
extremely rapid image deployment capabilities. It is also heavily dependent on the
built source image containing kexec support in the compiled in kernel.
22.26.1.9. Prerequisites¶
The image-builder content pack has Prerequisites necessary to operate correctly. In addition
to any content packs that define/describe BootEnvs and necessary workflow stages (for example
RHEL 7.7 requiers os-other
).
Please ensure you have the appropriate content packs to support workflows for building images as desired. The Preqrequisistes system does not call out content or plugins that are optional for building your image.
22.26.2. Object Specific Documentation¶
22.26.2.1. params¶
The content package provides the following params.
22.26.2.1.1. image-builder/additional-excludes¶
An array of strings, of files and/or directories to exclude from the
tar operation in the final image contents. These values are passed in as
exclude patterns, according to the tar
regex processing rules. Note
that these files/directories are not removed from the original image
built system, they are simply excluded from being rolled up in the tar.
Regex wildcards can be used, however, they are limited to shell glob
patterns as used by tar
.
The default values specified here should not be removed unless the operator is certain they need to be in place during the image deployment process.
Note
The tar
operations utilize a relative directory structure, rooted
in the root filesystem (/
). Adjust the file/directory paths
appropriately!! (eg ./etc/default/grub
, as opposed to /etc/default/grub
).
22.26.2.1.2. image-builder/additional-packages¶
During image deployment via the image-deploy
plugin, several actions are
required to prepare the system appropriately. This Param defines a list of
packages that are generally needed to ensure successful image deployment
Changing this list may make the deployed image faile; depending on the
requrested fileystem structures (curtin/partitions
), or hardware boot
modes (Legacy -vs- UEFI), etc.
If the operator intends to add to this list, the entire list will need to be re-iterated with the defaults plus the additionally required packages.
22.26.2.1.3. image-builder/additional-tar-options¶
An a string of additional tar
arguments to pass in to the image capture
process. It is critical that the operator understand where in the command
line flags this is injected and what current flags are passed in by default.
Please review the image-capture.sh.tmpl
template file for more details.
These are standard command line flags, and those rules apply; namely, properly formatted argument flags, separated by spaces.
An example usage might be to add various exclude arguments to prevent certain
files on the system from ending up in the deployed images (eg --exclude-backups
).
22.26.2.1.4. image-builder/skip-package-reset¶
A boolean value that allows the operator to skip the built in image-builder
package repository reset task. By default image-builder will reset packages
to the distro defaults. If the operator is using the package-repositories
feature, this will cause the defined repos to be wiped out.
The default value for this is false
. To skip repo reset, set this Param
on the machine, a profile on the machine, or the global profile to true
.
22.26.2.1.5. image-builder/upload-manifest-name¶
The final remote filename of the Manifest that is generated by the image-builder. This can utilize built in shell variables for building up a unique name. The only supported shell variables that are supported are:
$TAG
= The OS Name as defined in the BootEnv structure of the built image
$DSTAMP
= The Date Stamp fromdate +%Y%m%d-%H%M%S
RS_IMAGE_IDENT
= a uniquely generated ID with 10 character length (generated withuuidgen
andmd5sum
)
The name should not contain any filename extension, as a .txt
will be
automatically added. If the passed in value contains .txt
, it will
be ignored/preserved. If a filename extension other than .txt
is
provided in this Param, .txt
will still be appended to the final product.
The default value utilizes all of the optional shell variables that are supported and looks like:
$TAG-MANIFEST-$DSTAMP-$RS_IMAGE_IDENT
Note
“MANIFEST” is just upper case and not intended to be a shell variable in this example.
This will ultimately produce an uploaded file with the following format:
centos-8-MANIFEST-20201105-233755-53fcfe709c.txt
22.26.2.1.6. image-builder/upload-path¶
The remote path on the DRP Endpoint Files server, to upload the RootFS
image to. By default, the image will be uploaded to the images/
path, which would make it accessible at:
https://DRP:8092/files/images/IMAGE
or
http://DRP:8091/files/images/IMAGE
Replacing DRP with the name or IP address of the DRP Endpoint, and IMAGE with the upload image name.
Note
This path is relative to the files/
base path, and must not
contain any preceding slash (“/”). If it ends with a trailing
slash, it will be processed accordingly.
22.26.2.1.7. image-builder/upload-rootfs-name¶
The final remote filename of the built Image that is generated by the image-builder. This can utilize built in shell variables for building up a unique name. The only supported shell variables that are supported are:
$TAG
= The OS Name as defined in the BootEnv structure of the built image
$DSTAMP
= The Date Stamp fromdate +%Y%m%d-%H%M%S
RS_IMAGE_IDENT
= a uniquely generated ID with 10 character length (generated withuuidgen
andmd5sum
)
The name should not contain any filename extension, as a .tgz
will be
automatically added. If the passed in value contains .tgz
, it will
be ignored/preserved. If a filename extension other than .tgz
is
provided in this Param, .tgz
will still be appended to the final product.
The default value utilizes all of the optional shell variables that are supported and looks like:
$TAG-rootfs-$DSTAMP-$RS_IMAGE_IDENT
This will ultimately produce an uploaded file with the following format:
centos-8-rootfs-20201105-233755-53fcfe709c.tgz
22.26.2.2. stages¶
The content package provides the following stages.
22.26.2.2.1. centos-7-install-builder¶
This stage starts a CentOS 7 install for the Image Builder content.
It also will set the Repos based on the Install Repos specified for
CentOS 7 in the package-repositories
param.
22.26.2.2.2. finish-image-build¶
Used with the STOP runner action to leave Image Build host in wait state
22.26.2.2.3. image-builder-capture¶
This stage captures the built image as a RootFS tarball for deployment
by the image-deploy
plugin as a tgz
type.
Customization prior to this capture can be accomplished by setting the
Param image-builder/post-builder-flexiflow
to a list of additional
Tasks to execute.
22.26.2.2.4. image-builder-linux¶
An example stage that configures a CentOS 7 host to be ready for Image Builder to capture a root filesystem tarball image of the installed system.
Note that package repository handling may need to be changed for your environment, in which case you will want to construct a custom Stage that modifies repos correctly for your use.
In addition this Stage will update all packages to the “latest” in the package repositories. This may not be the correct behavior for your use case.
This stage performs the following functions:
Reset package repositories - reset package repos to system “stock”
Update all packages to the latest versions in the package repositories
Install
cloud-init
for image bootstrap configurationClean up the image by stripping out useless kernels, package meta files, etc.
With use of the image-builder/pre-builder-workflow
Paramater on the builder
machine, a dynamic list of Tasks can be injected BEFORE this Stage.
With use of the image-builder/post-builder-flexiflow
Parameter on the
builder machine, a dynamic list of Tasks can be injected AFTER this Stage,
but before the final image is captured.
22.26.2.2.5. image-builder-post-builder-flexiflow¶
This stage utilizes the Flexiflow content pack to allow image builds that can dynamically inject new tasks to customize the OS/image before cleanup.
To utilize this, please review the Flexiflow content for more complete details. Basic usage:
set the Param
image-builder/post-builder-flexiflow
as an Array of tasks to inject in place of this stage
Example:
image-builder/post-builder-flexiflow: - additional-cleanup
In our theoretical example above, we have 1 task. This task is run after the
image-builder-cleanup
task, but before the image is captured. It allows
the operator to inject additional tasks post-cleanup for any additional
preparation of the image just prior to capturing the image.
22.26.2.2.6. image-builder-pre-builder-flexiflow¶
This stage utilizes the Flexiflow content pack to allow image builds that can dynamically inject new tasks to customize the OS/image before cleanup.
To utilize this, please review the Flexiflow content for more complete details. Basic usage:
set the Param
image-builder/pre-builder-flexiflow
as an Array of tasks to inject in place of this stage
Example:
image-builder/pre-builder-flexiflow: - repository-add - install-my-packages - repository-remove
In our theoretical example above, we have 3 tasks. The first should add a new repository to the installed system (for example EPEL). The second would install some packages that are contained in the example repository. The third then removes the example repository from the sysmem - so the final installed system is in a clean state without those repos.
22.26.2.2.7. image-builder-start¶
This is a decorator stage that marks the beginning of the image builder process.
22.26.2.3. tasks¶
The content package provides the following tasks.
22.26.2.3.1. image-builder-cleanup¶
This task calls the image-builder-cleanup.sh.tmpl
Tempalte, which does
post-provisioning cleanup of the built image, prior to creating a root
filesystem tarball. Some of the actions performed include:
Perform package manager clean operations
remove rescue kernels/initrds from the image
zero out log files in
/var/log
empty the RackN
/etc/drpcli
filewipe contents of
/etc/resolv.conf
remove shell history
Please review the template carefully to insure the actions performed in the template are consistent with the documentation above, and are indeed the actions you want performed on your image.
22.26.2.3.2. image-builder-stage¶
Copies the built image to a remote HTTP file server location. Currently the only supported
HTTP file server is a DRP Endpoint, in the Files
location.
22.26.2.3.3. image-capture¶
The image-capture
task brands the image with detail information to properly
identify the built image version, temestamps, etc. The root filesystem will be
rolled up in to a Tar and GZip formatted archive (TGZ).
Last, the image will be uploaded to the DRP Endpoint in the HTTP/S Files file
server space for subsequent download and use by the image-deploy
plugin.
This task requires the ExtraClaims capability for the files
scope to be
able to authorize the Files upload. Older versions of this content required a
Param defined set of Username/Password values to access the DRP Endpoint files
upload capability.
22.26.2.3.4. image-install-cloud-init¶
Simple task to add cloud-init
pacakge to an Image Builder image
which can be used by the image-deploy
plugin to customize the installed
Machine.
22.26.2.3.5. image-reset-package-repos¶
Resets the system package configuraiton back to “factory default” values. During the image build process, it may be necessary to add in specific repos to add/configure software packages in to the gold master image. Once the image has been built, it is often desired to reset to a clean state as if the system had the default package configuration state.
22.26.2.3.6. image-update-packages¶
Should be run after fixing repo packages, but before installing cloud-init.
Installs additional required packages for successful deployment in various
image-deploy
scenarios.
22.26.2.4. workflows¶
The content package provides the following workflows.
22.26.2.4.1. image-builder-centos¶
This example workflow starts a CentOS Image Builder “gold image” build and capture process.
To create a customized version of the image for your use case, you may
want to clone this workflow and add stages between the centos-7-install
and image-builder-linux
stages. Any stage between those two will be
operating on the chroot root filesystem of the image that is built. Changes
will be part of your image.
22.26.2.4.2. image-builder-centos8¶
This example workflow starts a CentOS 8 Image Builder “gold image” build and capture process.
To create a customized version of the image for your use case, you may
want to clone this workflow and add stages between the centos-8-install
and image-builder-linux
stages. Any stage between those two will be
operating on the chroot root filesystem of the image that is built. Changes
will be part of your image.
22.26.2.4.3. image-builder-rhel-server¶
This example workflow starts a RHEL 7 Image Builder “gold image” build and capture process.
To create a customized version of the image for your use case, use the folowing stages to dynamically inject customization tasks in to this workflow:
image-builder-pre-builder-flexiflow
- runs beforeimage-builder-linux
stage
image-builder-post-builder-flexiflow
- runs afterimage-builder-linux
stage
See each of the stages for documentation on how to use them.
RHEL requires special handling to work due to how Redhat manages subscriptions. You may need to set the following Params:
image-builder/skip-package-reset: true redhat/subscription-username: "[Insert RHSM Username]" redhat/subscription-password: "[Insert RHSM Password]" redhat/rhsm-activation-key: "[Insert RHSP Activation Key]" redhat/rhsm-organization: "[Insert RHSM Organization ID]"
There are other redhat/rhsm-*
related Params that you may need in your environment.
Consult the list of Params for more configuration options.
22.26.2.4.4. image-builder-rhel8-server¶
This example workflow starts a RHEL 8 Image Builder “gold image” build and capture process.
To create a customized version of the image for your use case, use the folowing stages to dynamically inject customization tasks in to this workflow:
image-builder-pre-builder-flexiflow
- runs beforeimage-builder-linux
stage
image-builder-post-builder-flexiflow
- runs afterimage-builder-linux
stage
See each of the stages for documentation on how to use them.
RHEL requires special handling to work due to how Redhat manages subscriptions. You may need to set the following Params:
image-builder/skip-package-reset: true redhat/subscription-username: "[Insert RHSM Username]" redhat/subscription-password: "[Insert RHSM Password]" redhat/rhsm-activation-key: "[Insert RHSP Activation Key]" redhat/rhsm-organization: "[Insert RHSM Organization ID]" redhat/subscription-repos: - rhel-8-for-x86_64-appstream-rpms - rhel-8-for-x86_64-baseos-rpms - rhel-8-for-x86_64-supplementary-rpms
Note that the Subscription Repos must be set as the default values will not work for RHEL 8. The above three are known to work, but others may be added if needed. See the Param for more documentation on options.
There are other redhat/rhsm-*
related Params that you may need in your environment.
Consult the list of Params for more configuration options.
22.26.2.4.5. image-builder-rocky8¶
This example workflow starts a Rocky 8 Image Builder “gold image” build and capture process.
To create a customized version of the image for your use case, you may
want to clone this workflow and add stages between the rocky-8-install
and image-builder-linux
stages. Any stage between those two will be
operating on the chroot root filesystem of the image that is built. Changes
will be part of your image.
22.26.2.4.6. image-builder-ubuntu-bionic¶
This example workflow starts a Ubuntu 18.04 Image Builder “gold image” build and capture process.
To create a customized version of the image for your use case, you may
want to clone this workflow and add stages between the ubuntu-18.04-install
and image-builder-linux
stages. Any stage between those two will be
operating on the chroot root filesystem of the image that is built. Changes
will be part of your image.