Universal Hardware Architecture

RackN DRP provides hardware management and supports multiple levels of customization and control.

Hardware support is provided by the following plugins and content packs:

• flash
• bios
• raid
• ipmi

Additional hardware specific support is contributed by the content packs:

• hardware-tooling
• dell-support
• levovo-support
• hpe-support
• supermicro-support

These components provide stages, tasks, and parameters that provide the building blocks for hardware management.

The universal content pack takes these elements and combines them into pipeline elements that are consumed by the hardware provisioning pipelines.

The pipeline elements that use these pieces are universal-discover, universal-hardware, and universal-baseline. These elements can be injected into pipelines to enable hardware management.

Infrastructure Pipelines also have the ability to add Classification rules to dynamically add information (eg Profiles) to a system. All Machines that pass through the universal-discover Workflow can automatically have specially named Profiles automatically added to them. This pattern supports and enhances the hardware lifecycle management (HWLC) discussed in this document.

For further information, please see:

• Automatic Machine Classification in Infrastructure Pipelines

Functional Requirements

The combination of universal pipeline elements and hardware management components define a set of functions that are done as part of the management process. These fall into three groups.

• inventory
• configuration
• baseline
• reset

As we discuss each individual function, all functions should and do respect the following input parameters:

• rs-debug-enabled - optional boolean parameter - default false - If true, debug output should be generated for the task.

Each function has a set of contractual style input and output parameters that facilitate consistent usage outside of the normal operaiton of the function and allows follow tasks to use the results.

Inventory Functions

IPMI, Bios, Flash, and Raid all have inventory functions. These are called by the universal-discover pipeline element.

IPMI Inventory

IPMI inventory is used to detect if a BMC is present and its current networking configuration. This is controlled as part of the ipmi-inventory stage. The ipmi-inventory stage is a flexiflow stage that uses the ipmi/inventory-tasks parameter as the task list for the stage. The default task can be overridden by setting the ipmi/inventory-tasks parameter to other tasks for hardware types or environments.

The ipmi-inventory stage expects the following parameters:

Input Parameters:

• ipmi/prefer-v6 - optional boolean parameter - default false - If true, the system will attempt to use IPv6 for the calculated parameters.
• ipmi/force-protocol - optional string parameter - default unset - If preset, the value forces the source of the calculated parameters. Values: v4 or v6

Output Parameters:

• ipmi/enabled - Indicates the BMC is enabled. This is set to true if the BMC has an IP address.
• ipmi/macaddr - MAC Address of the BMC
• ipmi/address-v4 - IPv4 Address associated with the BMC
• ipmi/netmask-v4 - IPv4 Netmask associated with the BMC
• ipmi/ipsrc-v4 - Mode of operation of the BMC. Usually dhcp or static/manual
• ipmi/gateway-v4 - IPv4 Gateway address used by the BMC
• ipmi/address-v6 - IPv6 Address associated with the BMC
• ipmi/netmask-v6 - IPv6 Netmask associated with the BMC
• ipmi/ipsrc-v6 - Mode of operation of the BMC. Usually dhcp or static/manual
• ipmi/gateway-v6 - IPv6 Gateway address used by the BMC

These are directly gathered from the BMC.

Calculated Output Parameters:

• ipmi/address - The preferred IP address
• ipmi/netmask - The preferred netmask
• ipmi/ipsrc - The preferred ipsrc value
• ipmi/gateway - The preferred gateway value

The ipmi/force-protocol parameter is first precedence and the ipmi/prefer-v6 defines the second precedence.

The default task, ipmi-inventory, uses ipmitool to query the local system to gather information and does not use hardware specific tools.

If a custom task is used, it should respect the two IPMI specific to determine the values to assign the non-v4 and non-v6 parameters. These are used by other tasks and plugins to determine how to access the BMC. The output parameters should also be filled by the inventory task as well. ipmi/enabled is used by other IPMI tasks to determine if configuration operations should proceed.

BIOS Inventory

BIOS inventory is used to detect the current configuration of the BIOS values. This is controlled as part of the bios-inventory stage. The bios-inventory stage is a flexiflow stage that uses the bios-inventory-tasks parameter as the task list for the stage. The default task can be overridden by setting the bios-inventory-tasks parameter to other tasks for hardware types or environments.

The bios-inventory stage expects the following parameters:

Input Parameters: None

Output Parameters:

• bios-current-configuration - untype object - Used to store vendor specific hardware output information.

The stage is also expected to install required tools if not already present.

The default task, bios-current-config, uses the hardware-tooling and support content packs to install tools, set the bios-driver parameter to indicate hardware tooling, and gather the current configuration.

If a custom task is used, the system should record current state of the system BIOS configuration into the bios-current-config parameter. As a free form object, this data can take any form. Ideally, this format should be easily mappable into the bios-target-configuration parameter for consumption by the bios-configure task. The task should also ensure tools are installed by prerequiste reference.

RAID Inventory

RAID inventory is used to detect the current configuration of the RAID system. This is controlled as part of the raid-inventory stage. The raid-inventory stage is a flexiflow stage that uses the raid-inventory-tasks parameter as the task list for the stage. The default task can be overridden by setting the raid-inventory-tasks parameter to other tasks for hardware types or environments.

The raid-inventory stage expects the following parameters:

Input Parameters: None

Output Parameters:

• raid-current-config - untype object - Used to store vendor specific hardware output information.

The stage is also expected to install required tools if not already present.

The default task, raid-inventory, uses the raid-install-tools task to install tools and sets the raid-current-config parameter. Adittionally, the raid-available-utilities and the raid-usable-utilities parameters custom the tools to use. These two parameters are specific to the default tasks and should not be used in custom tasks.

If a custom task is used, the system should record current state of the system RAID configuration into the raid-current-config parameter. As a free form object, this data can take any form. Ideally, this format should be easily mappable into the raid-target-config parameter for consumption by the raid-configure and raid-enable-encryption task. The task should also ensure tools are installed by prerequiste reference.

Flash Inventory

Flash inventory is used to detect the current flash levels of the system components. This is controlled as part of the flash-inventory stage. The flash-inventory stage is a flexiflow stage that uses the flash-inventory-tasks parameter as the task list for the stage. The default tasks can be overridden by setting the flash-inventory-tasks parameter to other tasks for hardware types or environments.

The flash-inventory stage expects the following parameters:

Input Parameters: None

Output Parameters:

• flash-current-config - untype object - Used to store vendor specific hardware output information.

The stage is also expected to install required tools if not already present.

There is not a default task for this stage. It is a no-op.

If a custom task is used, the system should record current state of the system flash levels into the flash-current-config parameter. As a free form object, this data can take any form. Ideally, this format should be easily mappable into the flash-list parameter for consumption by the flash task. An intermediate step is probably expected for this task because mapping versions of flash to packages may be needed. The task should also ensure tools are installed by prerequiste reference.

Configuration Functions

The universal-hardware pipeline element uses the configuration functions to configure the components in questions. There are multiple configuration tasks depending upon the piece of the system.

IPMI Configuration

IPMI Install Certificate

BIOS Configuration

BIOS configuration is used to configure the BMC and other non-raid components of the system. This is controlled as part of the bios-configure stage. The bios-configure stage is a flexiflow stage that uses the bios-configure-tasks parameter as the task list for the stage. The default tasks can be overridden by setting the bios-configure-tasks parameter to other tasks for hardware types or environments.

The bios-configure stage expects the following parameters:

Input Parameters:

• bios-target-configuration - untype object - default empty object - This is used by the default task,but could be altered for other purposes. The format of the object is dependent upon the underlying task.
• bios-skip-config - boolean - default false - This parameters is used to skip bios. Some hardware types do not have an idempotent methodology and use the skip flag to keep from multiple runs. The reset task should clear this flag.
• bios-target-configuration-compose - boolean - default false - This parameter indicates if the bios configuration should be composed across all layers of configuration elements.
• bios-last-attempted-configuration - untype object - default empty object - Previously attempted bios configuration. This can be used to detect hardware setting failures.

Output Parameters:

• bios-last-attempted-configuration - untype object - default empty object - Updated to reflected attempted settting. This is cleared if nothing needs to be done.
• bios-current-config - untype object - default empty object - This could be updated to reflect the current configuration. This is not set by the default task currently.

The stage is also expected to install required tools if not already present.

The default task, bios-configure, uses the custom tool drp-bios to handle conversion of the normalized bios-target-configuration parameter into vendor specific patterns. This current task requires conversion of vendor specific formats into the bios-target-configuration parameter. This can be done with the universal-baseline pipeline, but requires manual edits. Alternative paths would be to use the generic tool runner to get more direct control of vendor tools directly using their formats. The default task controls reboots by task return code to allow for multiple pass settings.

If a custom task is used, it should follow the requirements. The task needs to use a reboot return code to reboot the system to allow for validation of configuration. If possible, the configure task should allow for multiple pass execution. This means that the current configuration should be difference with the target configuration and applied the difference. The reason is that some bios settings are conditionally available after other settings are set. This means that a multi-pass approach is required. Use the bios-target-confguration and the bios-last-attempted-configuration parameters set upon the machine to determine if progress is being made. If idempotent and multi-pass support can be used, use the bios-skip-config parameter to control successful one-pass operation.

RAID Configuration

RAID configuration is used to configure the RAID components of the system. This is controlled as part of the raid-configure stage. The raid-configure stage is a flexiflow stage that uses the raid-configure-tasks parameter as the task list for the stage. The default tasks can be overridden by setting the raid-configure-tasks parameter to other tasks for hardware types or environments.

The raid-configure stage expects the following parameters:

Input Parameters:

• raid-skip-config - boolean - default false - This parameter is used to skip raid configuration. The reset task should clear this flag. This flag is used to force skipping the task after successful configuration.
• raid-clear-config - boolean - default false - This parameter is used to indicate if the current configuration should be cleared before configuration. Generally, this is set to true, but additive configurations are possible.
• raid-target-config - typed array - default [] - This is the desired configuration across multiple controllers. See the parameter definition for type information.

Output Parameters:

• raid-current-config - untype object - default empty object - This could be updated to reflect the current configuration.

The stage is also expected to install required tools if not already present.

The default task, raid-configure, uses the custom tool drp-raid to handle conversion of the normalized raid-target-config parameter into vendor specific patterns. This current task requires conversion of vendor specific formats into the raid-target-config. This can be done with the universal-baseline pipeline, but requires manual edits. Alternative paths would be to use the generic tool runner to get more direct control of vendor tools directly using their formats. The default task assumes a single pass non-reboot path with the setting of raid-skip-config to true to indicate success and prevent future destructive actions to the disk.

If a custom task is used, it should follow the requirements. The task needs to use raid-skip-config to allow the system to prevent destructive actions. The task should attempt non-reboot single pass operations. If multiple passes are required, the raid-skip-config should be set to true only upon final successful execution of tasks.

RAID Enabled Encryption

RAID enable encryption is used to enable encryption components of the RAID system. This is controlled as part of the raid-enable-encryption stage. The raid-enable-encryption stage is a flexiflow stage that uses the raid-enable-encryption-tasks parameter as the task list for the stage. The default tasks can be overridden by setting the raid-enable-encryption-tasks parameter to other tasks for hardware types or environments.

The raid-enable-encryption stage expects the following parameters:

Input Parameters:

• raid-skip-encryption - boolean - default false - This parameter is used to skip raid encryption. The reset task should clear this flag. This flag is used to force skipping the task after successful configuration of encryption.
• raid-encryption-key - string - default unset - This parameter can specify the encryption key value.
• raid-encryption-password - string - default unset - This parameter can specify the encryption password value.
• raid-encryption-password-save - boolean - default false - indicates if the task should record built passwords

Output Parameters:

• raid-skip-encryption - boolean - set to true upon successful completion

The stage is also expected to install required tools if not already present.

The default task, raid-enable-encryption, uses the custom tool drp-raid to handle setting up encryption key/password settings for the controller. Alternative paths would be to use the generic tool runner to get more direct control of vendor tools directly using their formats. The default task assumes a single pass non-reboot path with the setting of raid-skip-encryption to true to indicate success and prevent future destructive actions to the disk.

If the raid-encryption-password is not set, a random password is created and optionally recorded in the raid-encryption-password field if the raid-encryption-password-save parameter is true.

If the raid-encryption-key is not set, the system serial number is used as the key for encryption.

If a custom task is used, it should follow the requirements. The task needs to use raid-skip-encryption to allow the system to prevent destructive actions. The task should attempt non-reboot single pass operations. If multiple passes are required, the raid-skip-encryption should be set to true only upon final successful execution of tasks. The same semantics around the key and password should be maintained if possible.

Flash Update

Flash is used to update the components of a system. This is controlled as part of the flash stage. The flash stage is a flexiflow stage that uses the flash-tasks parameter as the task list for the stage. The default tasks can be overridden by setting the flash-tasks parameter to other tasks for hardware types or environments.

The flash stage expects the following parameters:

Input Parameters:

• flash-method - string enum (list, block, or list-then-block) - default list-then-block - Should the system use a list of specific flash elements or the vendor block release methodology. list-then-block uses a list if the array is present otherwise uses the block method.
• flash-list - array - default unset - An array of vendor specific packages to apply to the system and what to do if they try to go backwards.
• flash-list-check-list - array - default unset - An array of already applied packages that can be skipped because they been applied in a previous pass.
• flash-list-force - boolean - default false - Should all the packages in the list be forced (the list can specify force independent elements)
• flash-override-manufacturer - string - default unset - This overrides the detected manufacturer to enable OEM tooling against OEMed gear.
• skip-flash - boolean - default false - should the flash system skip operations. The reset task should clear this flag.
• proxy-servers - array strings - default unset - proxy servers required to access the internet for remote packages

Output Parameters:

• flash-list-installed - array string - unset - The names of the install packages set upon success.
• flash-list-installed-hash - string - unset - An sha256 sum of the flash-list-installed parameter when set.

The stage is also expected to install required tools if not already present and enable repos for Sledgehammer if needed.

The default task, flash-discover, uses the flash-method to inject tasks to do a block update or a list update. A vendor specific tasks is injected to carry out the function. If the system can be idempotent, then flashing should be handled that way. In some cases, this is not possible and the skip-flash parameter should be set to true upon success. This parameter could also be set for performance reasons as well. Some tooling is very slow. The reset task should clear the skip-flash parameter.

Alternative paths would be to use the generic tool runner to get more direct control of vendor tools directly using their formats. This case would be a full tool replacement. For flash operations, an alternative update for systems that have packages or update tooling would be to add a block task and a list task that understands the underlying hardware and use the flash-override-manufacturer parameter to load those tasks. The tasks would be named <mfgr param value>-firmware-flash and <mfgr param value>-firmware-flash-list tasks, respectively.

Regardless of the path, the tools should allow proxy servers support so that remote files can be referenced directly from the managed system.

If a custom task is used, you should handle the requirement of rebooting by return code and rebooting BMCs as part of the process. Ideally, a list and block form would be provided.

Baseline Functions

The baseline functions are used by the universal-baseline pipeline element / pipeline to generate a baseline configuration profile. These profiles can be used to replicate configuration to other systems. Some manipulation of the resulting profiles will be needed to remove system specific information and convert flash inventories into flash packages.

IPMI Baseline

BIOS Baseline

RAID Baseline

Flash Baseline

Reset Functions

The reset functions are used by the universal-rebuild and universal-maintenance pipeline elements to clear flags and other parameters to enable the configuration tasks to do their jobs. The rebuild process is assumed to be systemically destructive so all subsystems are reset. The maintenance process is assumed to be systemically non-destructive so only the BIOS, IPMI, and Flash subsystems are reset.

IPMI Reset

BIOS Reset

RAID Reset

Flash Reset

Default Tooling

The default tooling uses the hardware-tooling content pack to provide basic support for Dell, HPE, Lenovo, and Supermicro hardware. RackN has attempted to maintain and update install tools and provide normalization across these tools. This is proving difficult to maintain as hardware vendor change and alter their tool chains across time and hardware platforms. The current tooling will continue to function as is, but may not support newer hardware releases or platforms.

The hardware-install task and its children tasks may be sufficient. For HPE, Dell, Lenovo, and Supermicro, the default tool installer attempts to get the latest tools and is configurable about what tools to install. Altering these parameters may be sufficient to install tools.

Hardware vendors other than the listed ones will need to make sure that their tools are installed during their provided custom tasks. This should be done by creating an idempotent install task that is a prerequiste of the functional task. This allows the system install the tools once and reuse them as needed.

Custom Tasks

Custom tasks may be needed to take advantage of newer hardware using newer tools. Each task should meet the defined contract and ensure the tools are installed.

Custom tasks could also include context changes to handle remote execution of container-based tools.

Generic Tool Runner

The generic tool runner is a set of tasks that can be placed into the system to replace the default tasks. The purpose of these tasks are to provide a profile-driven way to call installed tools in a consistent way so that customers do not have to write their own tasks, but can drive the vendor tools as appropriate.