22.29. image-deploy - Image Deploy¶
The following documentation is for Image Deploy (image-deploy) content package at version .
The image-deploy plugin deploys images to systems. These images can be tarballs that get untarred onto defined partition-based filesystems or raw disk images that get blown onto the disk. In either case, additional files can be injected into the images for post-install operations.
The provided stages and workflows also know how to drive cloud-init and provided the information required to let cloud-init startup the post-install process. This includes injecting DRPCLI-based runners.
The image-deploy system is built upon and extends the curtin project. Curtin is driven by a configuration YAML file that is built dynamically based upon the value of parameters. These parameters drive the actual imaging process.
### Getting Started
The ‘image-deploy-base’ Workflow has been defined as a getting started reference example. It contains the mose basic components needed for image provisioning.
## Image-Deploy Image Construction
image-deploy requires an image. This image can be in many different formats. The format type defines what additional operational parameters that need to be set. These images can be built using RackN’s image-builder content pack, using Hashicorp’s Packer tool (examples can be found here: FILL_IN), or other tools that generate images. The images have some specific requirements that must be met for success.
The first requirement is that all images must have a /curtin directory in the top-level directory of the base image file. If the image is a raw disk image, this directory must be backed into that. If the image is a rootfs tarball, the directory must be added as part of that tarball. This must be created by the base image or first image applied.
The second requirement is that all drivers must be added to the system in the base image or additional tarball packages. The image-deploy components do NOT handle driver updates or installs.
The third requirement is for raw disk images. The disk image must be constructed for the the boot-style of the target hardware. If the hard is going to boot in UEFI mode, then the disk image must be formatted with EFI partitions that hold EFI bootloaders. The same applies for legacy systems. There are disk layouts that can work for both.
A final item to consider during image construction is how will the next phase of operation start. This means what will run the next phase of operation. For windows, this could be an unattend.xml that is part of the image or injected by additional parameters. Or it could be cloud-init is installed and set to automatically start. The image-deploy system has the ability configure cloud-init meta-data and user-data with default to start a DRP runner on boot. The system will also inject scripts to install a runner for the OS type in question. These could also be called to install the agent as well. Depending upon the choice of startup method, different things need to be done. Some involve adding things to the image.
## Image-Deploy Operations
Once an image id built, the system needs to know where it is, what type of image it is, what OS it contains, and what startup process to use. These are all defined as parameters that can be added to a machine. Because all of this information is consistent across all machines using this image, it is best practice to create a profile that contains all these parameters.
To define the location of the image file, you must use ONE of the following parameters. If both are used, the system will use the image-deploy/image-file value.
image-deploy/image-file - This is a path relative to the tftpboot directory of the running DRP endpoint. e.g. If the image has been uploaded with the files API to images/my-win-image.raw.gz, then the value of this parameter would be files/images/my-win-image.tar.gz.
image-deploy/image-url - This is a URL that references the image. This must be an http or https URL that does NOT require authentication.
To define the type and content of the image, all of these values must be specified:
image-deploy/image-type - This is the curtin image type value. These types are define in the parameter and validated by the systems. Some examples are: dd-gz, tgz, and others.
image-deploy/image-os - This is the type of OS being installed. Current values are are linux or windows. The value of basic is deprecated and maps to windows.
image-deploy/image-os-subtype - This is the subtype of the image-deploy/image-os value. This is ignored currently for windows, but for linux, this is used to indicate a raw image. The raw value indicates that the disk is a full disk image and curtin should not run any hooks to alter the image state.
Additional parameters can be used to override and expand the system.
## Image-Deploy Customizations
With the basic parameters set in a profile, additional parameters can be added to drive further customization and enhancements.
### Partitioning
When using a raw disk image, partitioning is not used other than to select the hard disk. By default, this will be specified by the image-deploy/install-disk parameter with a value of /dev/sda. This value can be overridden to direct the install to a different location.
When using a rootfs and other tarballs, the system will use the image-deploy-install-disk as the default disk and use a simple GPT-based filesystem layout chosen by curtin. A custom partition table can be added by using the curtin/partitions parameter. This object-based parameter will be converted to yaml and injected into the curtin-config yaml.
The curtin/partitions parameter is an object that defines the disks, partitions, filesystems, and mount points for system. The definition of the object can be found here: <https://curtin.readthedocs.io/en/latest/topics/storage.html>.
### Additional Content
Once the base image and the RackN content tarball are added to the system, additional tarballs can be added to the main disk. These are defined in the image-deploy/additional-tarballs parameter. This is a list of objects that define file locations or URLs, and format (like the image-deploy/image-type parameter. The assumption is that these tarballs will be applied at the root level of the mounted images. These MUST be built prior to the curtin-deploy task is executed.
The image-deploy tasks build one tarball and apply it for all systems. This is the RackN Content tarball. This contains at a minimum an OS appropriate runner, the curtin hooks needed to complete installation if required, an install script to install the DRPCLI as a runner, and optionally cloud-init configuration pieces.
The following pieces are installed and their locations:
drpcli(.exe) - For linux, this is installed to /usr/local/bin. For windows, this is /curtin.
curtin hooks - These will go into /curtin and can be ignored.
For window’s systems, there is an additional file added if specified. If image-deploy/windows-unattend-template is defined, it should contain the name of a template that is loaded into the system. This template will be expanded and stored in the newly imaged filesystem at the location specified by image-deploy/windows-unattend-path. The default path is Windows/Panther/unattend.xml. This can be used to inject a dynamic unattend xml file for startup usage. This allows for a more flexible image.
For further flexibility, the system can be configured to configure cloud-init or cloud-base as well.
### Cloud-Init / Cloud-Base
The image-deploy system will attempt to configure cloud-init (or Cloud-Base the windows version of cloud-init) by default. To turn this off, set the image-deploy/use-cloud-init parameter to false.
For linux systems, the system will configure cloud-init to use a data source to find meta-data and user-data. These will be injected locally by default into the /curtin directory. The image-deploy/cloud-init-base-url parameter allows this to be overridden. This can be other directory locations and the files will be placed accordingly. The system also allows for a remote file location through specific URLs or parameter expansion. A common use would be to specific, {{.Machine.Url}}/. This expansion would map to files provided by the image-deploy-cloud-init stage.
For windows systems, the system will configure cloud-base to use the custom RackN data source that works similar to the data sources used by linux. The same parameters control the location and contents of the meta-data and user-data files.
The contents of the meta-data file includes the hostname, ssh access keys, admin-username, and admin-password. Hostname is specified by the Machine.Name field. The ssh access keys are pulled from the access-keys parameter. The admin-username and admin-password come from the parameters image-deploy/admin-username and image-deploy/admin-password respectively if specified.
The final piece of cloud-init data is the user-data file. If not specified, the default action will be to install and start the DRP runner as a last action of cloud-init. Other user-data content can be provided by filling in the string parameter, cloud-init/user-data. For best results, one should include a final run command to install and start the agent.
For linux,
For windows,
## Problem Determination
To debug issues, you can set the curtin/debug parameter to true and addtional curtin logs will be generated.
## Ubuntuhammer
As part of the 4.11 release RackN added support for a custom version of Ubuntu targeted directly at doing an image-deploy using Ubuntu as the OS instead of using Sledgehammer. Doing this allowed for the most recent version of curtin to be used. Changes to the curtin code made it too difficult to get it working on RHEL based distros, so using sledgehammer as the image-deploy deployment OS will lock the curtin version at 20.2 whereas using ubuntuhammer will provide 22.1 or newer. Ubuntuhammer will continue getting updates to its curtin version as new releases are made available, where sledgehammer will not.
For the 4.11 release Ubuntuhammer is available to be used in image-deploy, sledgehammer is still the default system used for image-deploy, but a new param was added to allow setting Ubuntuhammer as the image-deploy deployment OS. Using the opt-in method allows operators to try the new functionality without introducing changes to their current vetted processes.
To use Ubuntuhammer as the image-deploy deployment OS add the param image-deploy/deployment-os to a profile and set its value to “ubuntuhammer” then attach that profile to the machine. When you run image-deploy it should use ubuntuhammer instead of sledgehammer to deploy the image. This works like other params and can be in a profile directly attached to a machine, or at a global level by placing the param in the global profile. Another helpful global param is the kexec-ok param. Adding it to your global profile and setting the value to true will speed up the process by reducing unnecessary reboots.
22.29.1. Object Specific Documentation¶
22.29.1.1. params¶
The content package provides the following params.
22.29.1.1.1. cloud-init/user-data¶
This is the data to return to the node on cloud init calls.
This Param should contain either the cloud-config YAML user-data or a BASH script. You must pass content as a multi-line string, and it should conform to cloud-init correct syntax and formats. Reference cloud-config examples:
Note that several capabilities covered in cloud-config examples may be available directly in the image-deploy plugin via other Params configurations.
Digital Rebar Platform provides the user-data and metadata as a local set of files, written in to the deployed image. This is typically written in to the /curtin directory, where cloud-init will consume them.
22.29.1.1.2. cloud-init/x509-certs¶
This contains all the x509 certs for the system as a single string. Used to configure access to HTTPS (TLS Enabled) services that may not be recognized by as CA signed certificates.
22.29.1.1.3. curtin/debug¶
Enable enhanced debug output for curtin deployed installs via the image-deploy plugin.
22.29.1.1.4. curtin/debug-deploy-pause¶
If set to true, then the curtin-deploy task will pause for debugging the deployment environment. The pause will last a total of 30 minutes, after which provisioning will continue.
To force return to normal operation, remove the control file, and the pause loop will escape; continuing provisioning.
rm -f /tmp/curtin-debug-pause-deploy
22.29.1.1.5. curtin/network/bootmac¶
The MAC Address the system booted from. Used as the interface mac in the network config.
22.29.1.1.6. curtin/network/cidr¶
The numeric IP address and netmask for this Machine, in CIDR notation.
example: 10.10.10.10/24
22.29.1.1.7. curtin/network/gateway¶
The default gateway address for the network.
example: 10.10.10.1
22.29.1.1.8. curtin/network/ifname¶
The interface name to use inside the curtin network section.
Must be the same interface as the curtin/network/bootmac defined interface.
example: enp1s0f0
example: eth0
22.29.1.1.9. curtin/network/nameservers¶
A list of nameservers to use, in array format.
Example:
`json [ "1.1.1.1", "1.0.0.1", "8.8.8.8" ] `
22.29.1.1.10. curtin/network/template¶
A Digital Rebar Platform Template to include in to the cloud-config data to define the network configuration of the machine.
22.29.1.1.11. curtin/partitions¶
The partition configuration for a Machine which is deployed via the Root Tarball (tgz image type). You must include the storage, version, and config stanzas in the Param. See examples below for details.
This is a Curtin YAML format definition. For complete documentation on the data structure values and options, please see the documentation at:
If unspecified, the default action to setup a simple partition table suitable for linux rootfs installs (single / root filesystem as big as full disk size, no swapfile located in root as /swap.img).
Examples can be found in the Image Deploy Storage Examples. Here is an external link to the Examples (please note that this location may change in the future):
22.29.1.1.12. curtin/prereqs¶
List of packages needed to install and run curtin. Packages come from the image-deploy plugin and can be found at:
http://endpoint:8091/files/plugin_providers/image-deploy/*.rpm
See Schema default for an example.
22.29.1.1.13. curtin/skip-finalize-windows¶
Skip running the Windows finalize tasks. Default value is false (the finalize tasks SHOULD run).
22.29.1.1.14. curtin/version¶
The version of curtin to install and use. It comes from the image-deploy plugin. The version comes from http://endpoint:8091/files/plugin_providers/image-deploy/curtin-<version>.tar.gz
example: 20.2-rackn
example: 20.2-rackn3
example: master
22.29.1.1.15. id-test-index¶
If unset, the tests have not started. If set, it is the index of the current test.
22.29.1.1.16. id-test-tests¶
A list of profiles to test through.
22.29.1.1.17. image-deploy/additional-tarballs¶
A list of additional tarballs to install into the image during deployment time. This Param is an array of objects that define what tarballs to inject in to the final deployed image. This can be used to add configuration data, content, scripts, etc. inside the deployed system.
You can control the order in which multiple tarball files will be unrolled by setting the index value in a numerical order.
Example:
The values are defined as follows:
drpserved: will automatically fill in the full HTTP URI path to the DRP Files server if set to true. Do not specify a leading forward slash (/). If false, specify a full HTTP URI to the tarball file artifact.
format: a Curtin file format type (eg tar, or tgz). See Sources in note below.
uri: the URI of the resources, either a full HTTP URI path if drpserved is false, or a relative URI path with no leading forward slash, on the DRP HTTP Files server.
index: a unique numerical value to define the order in which the tarballs should be unrolled, as files in subsequent tarballs may potentially overwrite files in earlier tarballs.
- !!! note
Sources definition for Curtin configs is documented at: <https://curtin.readthedocs.io/en/latest/topics/config.html#sources>
22.29.1.1.18. image-deploy/admin-password¶
If specified, this can be used to set the admin password. This should be used in conjunction with image-deploy/admin-username.
If not specified, the default will be set to RocketSkates.
22.29.1.1.19. image-deploy/admin-username¶
If specified, this can be used to set the admin username. This should be used in conjunction with image-deploy/admin-password.
If not specified, the default will be set to rocketskates.
22.29.1.1.20. image-deploy/cloud-init-base-url¶
This parameter defines the base path to the user-data and meta-data files. The valid values are file:///path, http[s]://hostname:port/path, or just path.
The default is for local mode, file:///curtin/.
For remote usage, the value should be {{.Machine.Url}}/. This
works with the image-deploy-cloud-init stage.
The path MUST end in a slash.
22.29.1.1.21. image-deploy/deployment-os¶
The value here defaults to sledgehammer. RackN provides an alternative OS just for image-deploy called ubuntuhammer, and the value ubuntuhammer could be used as well.
22.29.1.1.22. image-deploy/image-file¶
DEPRECATED: image-deploy/image-url should be used instead. The format of the value should be {{.ProvisionerURL}}/this-current-value.
example: {{.ProvisionerURL}}/isos/centos-7.tgz
example: {{.ProvisionerURL}}/files/images/windows/windows-server-2019.raw.gz
example: {{.ProvisionerURL}}/my_dir/some_stuff/ubuntu.tgz
The file image for image-deploy to install. This will have the Provisioner’s URL prepended to the filename.
You may use the drpcli isos upload … command to place an image into the system. If this method is used, please prepend ‘isos/’ to your filename. You MUST NOT have any other directories in the filename; they must reside in the base isos/ directory.
example: isos/centos-7.tgz
You may use the drpcli files upload … command to place an image into the system. If this method is used, prepend ‘files/’ to your filename. You MAY use additional directories in the filename.
example: files/images/windows/windows-server-2019.raw.gz
You may also just place the image file under the DRP tftpboot directory. If this method is used, use the path relative to the tftpboot directory.
example: my_dir/some_stuff/ubuntu.tgz
In the above example, this might resolve to the following path on the Digital Rebar Platform filesystem, based on a default “production” install type:
/var/lib/dr-provision/tftpboot/my_dir/some_stuff/
22.29.1.1.23. image-deploy/image-installed-size¶
Defines the installed size required for the image when installed to a disk. Images can be compressed, and this will make it hard to know what the required disk size is for the installed OS.
If set, the value of this Param will be used to check against the reported disk size that the image will be deployed to. If the disk is too small, the validation task will fail, and the Image Deploy process will error out.
The String values should be a number followed by M or G to represent the value in Megabyte (1024 kilobytes), or Gigabytes (1024 megabytes). Do not use a space between the value and size designator.
Examples:
500M: 500 Megabyte disk required (512,000 kilobyte or bigger)
60G: 60 Gigabyte disk required (61,440 megabyte or bigger)
The validation schema only supports 1 to 10 total characters followed by M or G. This allows for up to 9,999,999,999 Megabyte or 9,999,999,999 Gigabyte disk sizes.
22.29.1.1.24. image-deploy/image-os¶
The file image contains the install contents for an operating system. This variable defines the image OS at a high level.
Valid values are:
esxi
linux
windows
- !!! note
The type basic has been deprecated and is no longer supported.
22.29.1.1.25. image-deploy/image-os-subtype¶
While image-deploy/image-os defines the overall OS type. This variable can be used to define the sub-type.
This is really only used for linux and it supports raw and other.
raw is used to support a raw image that does not need hooks.
other is used to attempt to support other linux (not debian-based or redhat-based).
22.29.1.1.26. image-deploy/image-type¶
The file image may be in many formats. This tells image-deploy what method to use to install.
Validated methods are:
tgz - Tar gzip compress filesystem
dd-tgz - tar file containing a single gzip compressed disk image
dd-txz - tar file containing a single XV compressed disk image`
dd-tbz - tar file containing a single BZ compressed disk image`
dd-tar - tar file containing a single uncompressed disk image`
dd-bz2 - tar file containing a single BZ2 compressed disk image`
dd-gz - raw disk image that is gzip compressed
dd-xz - raw disk image that is XZ compressed`
dd-raw - raw disk image that is uncompressed
- !!! note
For the most up to date list, see the Param type definition below on your DRP Endpoint.
22.29.1.1.27. image-deploy/image-url¶
The URL location for the image for image-deploy is to install. This must be a URL that is accessible from the Machine being deployed at the time of deployment.
22.29.1.1.28. image-deploy/install-disk¶
The disk that the image-deploy system should install to. The default value is /dev/sda. If the disks in the system are iterated with different device names, then you must adjust this Param appropriately. For example, virtual machines may iterate their disks to the Sledgehammer environment as /dev/vda, so you would set this Param value to /dev/vda in that case.
For troubleshooting, boot the Machine in to Sledgehammer (eg via the discover-base workflow; or any other appropriate workflow), and use lsblk to iterate and identify the device names of the disks.
22.29.1.1.29. image-deploy/other-dracut-path¶
The path to the dracut executable used by the linux/other os-type/os-sub-type pair.
22.29.1.1.30. image-deploy/other-initrd-path¶
The path to the dracut file that dracut should create.
The ${kv} string should be used to indicate the version of the kernel.
22.29.1.1.31. image-deploy/remove-mdev-text-files-from-dev¶
Ubuntuhammer and Sledgehammmer have mdev.seq and mdev.log files in the dev directory. This causes a problem with curtin when it attempts to builds its dev list. In sledgehammer we patched the curtin code to work around it, and in ubuntuhammer we use this variable to remove the files. If for some reason you need the files, set this param to false in your image-deploy profile attached to the machine.
22.29.1.1.32. image-deploy/skip-fstab¶
Skip rewriting the fstab on the target image.
22.29.1.1.33. image-deploy/skip-grub2-configure¶
Whether to skip trying to reconfigure grub.
If you are deploying a raw image type, you MUST set this value to true as the raw image already (should) contain a valid boot loader installed.
22.29.1.1.34. image-deploy/skip-network-configure¶
Whether to skip basic network configuration. Should be set to true if your image already has a working network interface configuration.
22.29.1.1.35. image-deploy/swapfile-maxsize¶
The maximum size of the curtin created swap file. This will default to 8GB if unspecified.
The value of this string is a size string, ##KB, ##MB, ##GB, or ##TB. The B is optional.
22.29.1.1.36. image-deploy/swapfile-name¶
The path to the linux swap file created by curtin.
If unspecified, the path will default to /swap.img.
22.29.1.1.37. image-deploy/swapfile-size¶
The exact size of the curtin created swap file. This will default to something upto the max size.
The default if unspecified is to create a file bounded by the value of image-deploy/swapfile-maxsize or 8GB.
The value of this string is a size string, ##KB, ##MB, ##GB, or ##TB. The B is optional.
To turn off creation of the swapfile, set this value to 0.
22.29.1.1.38. image-deploy/use-cloud-init¶
Set to true (the default) for the image deploy system to attempt to configure cloud-init to use a meta-data/user-data pair of files. These rely on other paraemeters to fill in the secondary files.
The type of cloud-init access is controlled by the parameter, image-deploy/cloud-init-service.
22.29.1.1.39. image-deploy/windows-license-key¶
The license key for the windows operating system as a string.
22.29.1.1.40. image-deploy/windows-unattend-path¶
The path to the Windows unattend.xml file. The default value is Windows/Panther/unattend.xml.
22.29.1.1.41. image-deploy/windows-unattend-template¶
The template to use for unattend.xml file.
22.29.1.2. stages¶
The content package provides the following stages.
22.29.1.2.1. id-test-init-test¶
Run another test stage
22.29.1.2.2. id-test-next-test¶
Run another test stage
22.29.1.2.3. image-deploy¶
Deploy an image within sledgehammer
22.29.1.2.4. image-deploy-cloud-init¶
Stage to handle serving a cloud-init file
22.29.1.2.5. image-deploy-no-erase¶
Deploy an image within sledgehammer without wiping disks first
22.29.1.3. tasks¶
The content package provides the following tasks.
22.29.1.3.1. cloud-init-wait¶
This will cause the system to wait for cloud-init to finish and start the runner before advancing to far into the task list.
The task works for linux, darwin, and windows OSes.
22.29.1.3.2. curtin-deploy¶
A task to install an image through curtin.
22.29.1.3.3. curtin-install¶
A task to install curtin.
22.29.1.3.4. id-test-init-test¶
A task to pick the init test.
22.29.1.3.5. id-test-next-test¶
A task to pick the next test.
22.29.1.4. workflows¶
The content package provides the following workflows.
22.29.1.4.1. image-deploy-base¶
Provides the minimal workflow needed to install an image using the image deploy process. Cloud-init defaults includes starting the runner and ssh-access.
The three Params are required
Location of image: either image-deploy/image-file or image-deploy/image-url
Operating system: image-deploy/image-os
Image format: image-deploy/image-type
