Provision¶

GREG: change this into the object pieces. TODO: Is this still necessary?

Provisioning Models¶

These models work together to manage any and all lifecycle needs for managing machines in DRP. This includes:

Keeping track of what machines are being managed by DRP.
Controlling what OS environment any given machine will boot to over the network.
Managing the order in which tasks will be run on machines.
Managing the context in which machine tasks will be run.
Making sure that any files that are needed to complete the provisioning process are available and valid.

Template¶

Template expansion underlies just about everything that DRP does. All template expansion in DRP happens accroding to the rules defined by the golang text/template package, which (along with this document) is required reading if you want to build content for DRP. Template objects define common template content that different parts of DRP can reuse as they see fit.

TemplateInfo¶

Closely related to the template is the TemplateInfo object, which is included as part of the bootenv, stage, and task objects.

Field	Definition
`Name`	The name of the template info.
`Path`	A string that will be expanded as if it were a template to generate a path for the template.
`Contents`	If present, a string that will be expanded as if it were a template to generate the file that will be made available at the location indicated by the `Path` field. Contents must be empty or not present if `ID` is set.
`ID`	If present, the ID of the template that will be used to generated the file that will be made available at the location indicated by the `Path` field. `ID` must be empty or not present if `Contents` is set.

Rendering Templates¶

Whenever DRP needs to render something as a template (whether it is a Template object, a TemplateInfo object, or just a string that might contain a template), it always does so in the context of a RenderData object, which provides a slew of useful helper functions along with references to the applicable objects. Render data is what DRP uses for dot or {{ . }} when executing a template.

Param¶

Params are how DRP provides validation and a last-ditch default value for data that we use during template expansion. You do not have to define a param in order to use it during template expansion, but DRP will not be able to enforce that param data is syntactically valid.

Secure Params¶

Secure param management is a licensed feature. You must have a license with the secure-params feature enabled to be able to create and retrieve secure param values. SecureData uses a simple encryption mechanism based on the NACL Box API (as implemented by libsodium, golang.org/x/crypto/nacl/box, tweetnacl-js, PyNaCl, and many others), using curve25519 and xsalsa20 for crypto, and poly1305 for message verification.

Secure params are handled by the API and stored on the backend using a SecureData struct, which has the following fields:

Field	Definition
`Payload`	The encrypted payload. When marshalled to JSON, this should be converted to a base64 encoded string.
`Nonce`	24 cryptographically random bytes. When marshalled to JSON, this should be converted into a base64 encoded string.
`Key`	A 32 byte curve25519 ephemeral public key. When marshalled to JSON, this should be converted to a base64 encoded string.

When a Param has the Secure flag, the following additional steps must be taken to set and get values for this param on objects that hold params.

Setting Secure Param Values¶

Get the peer public key for the object you want to set a secure param on from its pubkey endpoint. These endpoints are at /api/v3/<objectType>/<objectID>/pubkey. As an example, the pubkey endpoint for the global profile is /api/v3/profiles/global/pubkey. Access to these API endpoints requires an appropriate claim with the updateSecure action. These API endpoints return a JSON string containing the base64 encoding of an array containing 32 bytes.
Generate local, ephemeral curve25519 public and private keys using a cryptographically secure random number source.
Generate a 24 byte nonce using a cryptographically secure random number source.
Encrypt the JSON-marshalled param using the nonce, the peer public key, and the ephemeral private key.
Generate a SecureData struct with Key set to the ephemeral public key, Nonce set to the generated nonce, and Payload set to the encrypted data.
Use the SecureData struct in place of the raw param value when making API calls to add, set, or update params.

Retrieving Decrypted Secure Data Values¶

In order to retrieve decrypted secure data values, you must have an appropriate claim with the getSecure action. That will allow you to make GET requests to the params API endpoints for param-carrying objects with the decode=true query parameter. That will cause the frontend to decrypt any encryped parameter values before returning from the API call.

Rendering a Task for a Machine¶

The templates for a task are rendered for a specific machine whenever the actions for the job for that particular task/machine combo are requested.

All referenced templates can refer to each other by their ID (if referring to a template object directly), or by the template info name (if the TemplateInfo object), in addition to all the template objects by ID.

Template Prerequisite Expansion¶

When a task is added to a task list, its fully expanded list of prerequisite tasks are expanded, any tasks in that expanded list that already appear in the machine task list in the same bootenv are discarded, and the resultant set of prerequisite tasks are inserted just before the task to be inserted.

Profile¶

Profiles are named collections of parameters that can be used to provide common sets of parameters across multiple machines.

Stage¶

The stage for a machine is rendered after DRP starts up, whenever a machine changes to a different stage, or whenever a stage referred to by a machine changes.

All of the templates referred to by the Templates section of the stage will be rendered as static files available over the http and tftp services of the provisioner at the paths indicated by each entry in the Templates section. All referenced templates can refer to each other by their ID (if referring to a template object directly), or by the template info Name (if the TemplateInfo object), in addition to all the template objects by ID.

BootEnv¶

Boot environments (bootenv) are what DRP uses to model a network boot environment.

Rendering the unknownBootEnv¶

The bootenv for the unknownBootEnv preference is rendered whenever DRP starts up or the bootenv for the preference is changed. It is the only time that templates are rendered without a machine being referenced, which is why bootenvs that can be rendered this way must have the OnlyUnknown flag set.

All of the templates referred to by the Templates section of the bootenv will be rendered as static files available over the HTTP and TFTP services of DRP at the paths indicated by each entry. All referenced templates can refer to each other by their ID (if referring to a template object directly), or by the templateinfo Name (if the templateinfo object) in addition to all the template objects by ID.

Rendering a BootEnv for a Machine¶

The bootEnv for a machine is rendered whenever DRP starts up, whenever a machine changes to a different boot environment, or whenever a boot environment referred to by a machine changes.

All of the templates referred to by the Templates section of the bootenv will be rendered as static files available over the HTTP and TFTP services of DRP at the paths indicated by each entry in the templates section. All referenced templates can refer to each other by their ID (if referring to a template object directly), or by the templateinfo name (if the templateinfo object), in addition to all the template objects by ID.

Customization using bootenv-customize¶

DRP allows for programmatic overrides of various aspects of different bootenvs on a per-machine (via the usual parameter inheritance mechanisms) or global (via overides in the global profile) basis.

This param allows you to overlay dynamic customizations on top of bootenvs. Its intended use is to reduce the number of bootenvs you have to create for what are ultimately trivial reasons. This param is structured as a map of bootenv name to override values. You can override the following bootenv fields:

Loaders
OS
Templates

The override values have the same names and semantics of the equivalent fields in the bootenv, with the following exceptions:

The Templates section is merged in with the Templates section from the bootenv, with identically-named templates from this param overriding the ones from the bootEnv.
You must use the per-arch SupportedArchitectures section to override kernel, initrd, iso, boot parameter, and boot loader values from the bootenv.

Note

This is not compatible with bootenv purging. You may experience UI pauses when uploading a new ISO file. You will also get unexpected results if you create multiple overrides for the same bootenv that have the same OS name but reference different ISOs, or have different kernel/initrd settings. You should also not try to override sledgehammer on a per machine basis. Any overrides for sledgehammer must be matched by identical changes in the OS section of the discovery bootenv.

Cluster and Resource Broker¶

The API provides two simulated objects: "cluster" and "resource broker". These are special machine objects that gain a parallel profile that is named the same and follows the same lifecycle of the machine.

The cluster is intended to represent a collection of things that need workflows or work orders run against it.

The resource broker is intended to be a source of machines. Using a workflow or work order, the resource broker constructs machines and adds them to the system with an set of controls to add profiles and workflows to the create machines. The resource broker can be used to clean-up those machines as well.

Task Control Operations¶

To help with task and workflow debugging or advanced flow control, there are a couple of parameters that can help with debug messages and task control.

Single Step¶

Setting the machine parameter task-single-step to true will cause the system to make the machine not runnable after each task. Removing the parameter or setting it to false will cause the system to resume normal operations.

Stop At¶

Setting the machine parameter task-stop-at to a list of tasks or action entries will cause the system to stop prior to those tasks. Setting the machine to runnable will cause the tasks to continue.

Retry¶

Setting the machine parameter task-retry to a map of key/value pairs where the keys are task names and the values are integer number of times to retry the failing task. The system will retry that number of times before failing. The system will cause an exponential backoff starting at 2 seconds between retries. There is no maximum wait time.

The machine field RetryTaskAttempt tracks the retry for the current task. This is reset to 0 for every new task. For example, if the flaky task is always-fails then setting the task-retry parameter to 3 will cause DRP to retry the task three times.

Error Handlers¶

Setting the Machine parameter task-error-handlers to a map of key/value pairs where the keys are task names and the values are lists of task names. When a task fails and all the retries have been attempted, the map will be consulted and if present, the task list will be executed to and if successful, the failing task will be restarted and retried again with a reset RetryTaskAttempt. Error handling tasks can not be error handled.

The Machine field TaskErrorStacks is used to track the previous task list and current task to return to after running the error lists.

Parameter Conversion¶

Setting the machine parameter task-parameter-conversion to a map of key/value pairs where the keys are parameter names that should reference the value as a parameter name.

When the task is rendered for a machine or work order, a referenced parameter will be looked up in the task-parameter-conversion map and the new parameter name will be used for supplying a value. The lookup is recursive and will continue checking the map until a matching parameter is not found.

The task-parameter-conversion is composed and expanded before the lookups are done. The default map is empty.