Skip to content

Openshift

OpenShift Content Bundle

This content pack provides comprehensive tooling and automation for deploying and managing OpenShift clusters through Digital Rebar Platform (DRP). It handles the complete lifecycle of OpenShift clusters including installation, configuration, node management, and advanced features like OpenShift Virtualization (OCPV).

For Advanced Cluster Management, see the OpenShift Advanced Cluster Management content bundle.

Design Philosophy

The content bundle is designed around several key principles:

  1. Pipeline-Driven Deployment: The main cluster deployment is handled through a specialized profile (pipeline) that orchestrates the entire process. This ensures consistency and reduces human error.

  2. Task-Based Management: Individual administrative tasks are packaged as blueprints, allowing for targeted operations to manage cluster.

  3. Flexible Infrastructure: Support for both DRP-managed and external DNS, disconnected installations, and various infrastructure configurations.

  4. Automated Coordination: Tasks like node approval and cluster joining are automatically synchronized to ensure proper cluster formation.

Architecture

Node Types

The content bundle supports four distinct node types:

  1. Bootstrap Node
  2. Temporary node that initializes the cluster
  3. Minimum 2 vCPUs, 8GB RAM, 100GB disk
  4. Converts to worker node after cluster initialization

  5. Provides initial control plane services

  6. Control Plane Nodes

  7. Manage cluster's core services (API server, scheduler, etcd)
  8. Minimum 4 vCPUs, 16GB RAM, 100GB disk per node
  9. Requires exactly three nodes for production

  10. Must have identical hardware specifications

  11. Worker Nodes

  12. Run application workloads and containers
  13. Minimum 2 vCPUs, 8GB RAM, 100GB disk
  14. Scalable based on workload demands

  15. Can have varying hardware specifications

  16. Load Balancer Nodes

  17. HAProxy-based traffic distribution
  18. Minimum 2 vCPUs, 4GB RAM, 20GB disk
  19. Multiple nodes recommended for HA
  20. Handles API and application ingress

Network Architecture

The cluster uses three distinct network segments that MUST NOT overlap:

  1. Machine Network (Default: 172.21.0.0/20)
  2. Used for node IP addresses
  3. Must be routable within infrastructure

  4. Hosts API endpoints and load balancers

  5. Service Network (Default: 172.30.0.0/16)

  6. Used for Kubernetes services
  7. Internal cluster communications

  8. Not routable outside cluster

  9. Cluster Network (Default: 10.128.0.0/14)

  10. Pod networking
  11. Configurable host prefix (default: /23 - 512 pods per node)
  12. Internal container communication

Prerequisites

Infrastructure Requirements

  • DNS configuration (two options):
  • DRP-managed DNS (default): DRP automatically manages required DNS records
  • External DNS: Must manually configure DNS records as detailed in the DNS configuration section
  • Network connectivity between all nodes
  • Internet access or configured disconnected registry
  • Valid Red Hat OpenShift subscription
  • Sufficient network capacity for cluster traffic

Required Parameters

  • broker/name: Resource broker name (typically "pool-broker" for pool-based deployments)
  • openshift/pull-secret: Red Hat registry authentication (obtain from Red Hat OpenShift Cluster Manager)
  • openshift/cluster-domain: Base domain for cluster DNS

Optional Parameters

  • openshift/workers/names: Worker node hostnames
  • openshift/controlplanes/names: Control plane node hostnames
  • openshift/bootstraps/names: Bootstrap node hostname
  • openshift/load-balancers/names: Load balancer hostnames
  • openshift/external-registry: Disconnected registry configuration

Required Files

The following files must be accessible to DRP:

  • OpenShift Installer:
  • Download from: https://mirror.openshift.com/pub/openshift-v4/amd64/clients/ocp/stable-4.15/
  • File: openshift-install-linux.tar.gz or version-specific openshift-install-linux-4.15.46.tar.gz
  • Upload to DRP at: /files/redhat/openshift/openshift-install-linux-4.15.6.tar.gz

  • Param: openshift/installer-url

  • OpenShift Client Tools:

  • Download from: https://mirror.openshift.com/pub/openshift-v4/amd64/clients/ocp/stable-4.15/
  • File: openshift-client-linux.tar.gz or version-specific openshift-client-linux-4.15.46.tar.gz (includes oc and kubectl)
  • Upload to DRP: /files/redhat/openshift/oc-4.15.6-linux.tar.gz

  • Param: openshift/oc-url

  • OpenShift Mirror Tool (for disconnected installations):

  • Download from: https://mirror.openshift.com/pub/openshift-v4/amd64/clients/ocp/stable-4.15/
  • File: oc-mirror.tar.gz
  • Upload to DRP at: /files/redhat/openshift/oc-mirror.rhel9.tar.gz
  • Param: openshift/oc-mirror-url

ISOs

  • RedHat CoreOS - RHCOS
  • Download from: https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/
  • File: rhcos-4.15.23-x86_64-live.x86_64.iso
  • RHCOS
  • Dowload from: https://github.com/vmware/photon/wiki/Downloading-Photon-OS
  • File: https://packages.vmware.com/photon/5.0/GA/iso/photon-5.0-dde71ec57.x86_64.iso

Containers

  • Fedora container for oc-context

Version Customization

The bootstrap-openshift-versions bootstrap profile can be applied to a self-runner machine along with the bootstrap-versions param to specify which versions of openshift to download artifacts for and install. This bootstrap operation will automatically run a script similar to the custom bundles below.

Custom Bundles

The bundle_custom_version.sh tool generates a content bundle containing a bootstrap and cluster profile for a desired version of openshift.

OS_VERSION=latest-4.16
ops-tools/bundle_custom_version.sh $OS_VERSION >openshift-$OS_VERSION.yaml
drpcli contents upload openshift-$OS_VERSION.yaml

The bootstrap-openshift-<version> profile can be added to the self-runner to download the needed artifacts when running the rebootstrap-drp blueprint or the universal-bootstrap workflow.

The openshift-cluster-<version> profile can be added during cluster creation to specify which version of openshift and coreos to install.

Airgap

All required files can be downloaded using the download_airgap_artifacts.sh tool. This tool also provides the DRPCLI file and iso commands needed to upload the required files.

ops-tools/download_airgap_artifacts.sh 4.15.46

Deployment Process

The deployment is orchestrated by the universal-application-openshift-cluster pipeline, which is implemented as a specialized DRP profile. The process can be initiated through either the DRP web interface or CLI.

Web Interface Deployment

  1. Navigate to the cluster wizard
  2. Click "Add +" to create a new cluster
  3. Select "openshift-cluster" as the Cluster Pipeline
  4. Select "oc-cluster" as the context
  5. Select appropriate broker (typically "pool-broker")
  6. Paste your pull secret and click "Save"

CLI Deployment

The following assumes your pull-secret is stored in the global profile.

# Create cluster configuration
cat > cluster-config.yaml <<EOF
---
Name: demo
Profiles:
  - universal-application-openshift-cluster
Workflow: universal-start
Meta:
  BaseContext: oc-cluster
Params:
  broker/name: pool-broker
EOF

# Create the cluster
drpcli clusters create - < cluster-config.yaml

Deployment Stages

The deployment process consists of three main phases:

  1. Pre-provisioning Tasks: 

    universal/cluster-provision-pre-flexiflow:
  - openshift-cluster-tools         # Install OpenShift CLI and required tools
  - openshift-cluster-external-registry-create  # Setup disconnected registry if configured
  - openshift-cluster-external-registry-update  # Mirror required images if using disconnected registry
  - openshift-cluster-prep         # Generate cluster configuration and ignition files

  2. Resource Provisioning:

  3. The resource broker (typically pool-broker) selects or creates the required machines
  4. Machines are assigned appropriate roles (bootstrap, control plane, worker, load balancer)
  5. Base operating system is installed and configured

  6. Nodes wait at the approval stage for orchestrated deployment

  7. Post-provisioning Tasks: 

    universal/cluster-provision-post-flexiflow:
  - openshift-cluster-join    # Orchestrates cluster formation and node joining

The pipeline ensures these phases execute in the correct order and handles all necessary synchronization between nodes.

Testing OpenShift

Deploy Test Application

# Create a new project
oc new-project hello-openshift

# Create the deployment
kubectl create deployment hello-node --image=registry.k8s.io/e2e-test-images/agnhost:2.43 -- /agnhost serve-hostname

# Expose the service
oc expose deployment hello-node --port=9376
oc expose service hello-node

# Test the deployment
curl hello-node-hello-openshift.apps.demo.k8s.local

# Scale the deployment
oc scale deployment hello-node --replicas=3

# Cleanup (removes all resources in the project)
oc delete project hello-openshift

Advanced Features

Disconnected Installations

Support for air-gapped environments through: - External registry configuration - Image mirroring capabilities - Certificate management - Custom catalog sources

Load Balancer Configuration

By default, the content bundle configures HAProxy for cluster load balancing. However, production deployments often use external load balancers. Regardless of the implementation, the following ports must be configured:

  • API server (port 6443)
  • Machine config server (port 22623)
  • HTTP ingress (port 80)
  • HTTPS ingress (port 443)

The load balancer configuration works in conjunction with the DNS configuration to provide access to cluster services.

Administrative Tasks

The content bundle includes several blueprints for common administrative tasks:

  • openshift-cluster-status: Check cluster health and components
  • openshift-cluster-dns-refresh: Update DNS and load balancer configuration
  • openshift-cluster-remove-node: Safely remove nodes from the cluster

Troubleshooting

Common Commands

# Check node status
oc get nodes

# View cluster operators
oc get clusteroperators

# Monitor pod status
oc get pods --all-namespaces

# Check events
oc get events --sort-by='.metadata.creationTimestamp'

# View cluster version
oc get clusterversion

# List available upgrade versions
oc adm upgrade

# Initiate upgrade
oc adm upgrade --to=<version-number>
# Example: oc adm upgrade --to=4.15.36

Resource Cleanup

Dedicated tasks for cleanup operations: - openshift-cluster-cleanup: General cluster cleanup

DNS Configuration

When using external DNS, the following records must be configured (example for cluster "demo.k8s.local"). All records should use TTL of 0.

Name Type Value
ns1 A \
smtp A \
helper A \
helper.demo A \
api.demo A \
api-int.demo A \
*.apps.demo A \
cp1.demo A \
cp2.demo A \
cp3.demo A \
worker1.demo A \
worker2.demo A \
worker3.demo A \

Local DNS Configuration

When using DRP as the DNS host, configure a client to use DRP as the DNS host for the kubernetes doamin:

Add-DnsClientNrptRule -Namespace ".k8s.local" -NameServers "192.168.100.1"
Clear-DnsClientCache

To remove the rule

# Get the rule ID first
$rules = Get-DnsClientNrptRule | Where-Object {$_.Namespace -eq ".k8s.local"}
# Remove the rule using its ID
Remove-DnsClientNrptRule -Name $rules[0].Name

Version Customization

The bootstrap-openshift-versions bootstrap profile can be applied to a self-runner machine along with the bootstrap-versions param to specify which versions of openshift to download artifacts for and install. This bootstrap operation will automatically run a script similar to the custom bundles below.

Custom Bundles

The bundle_custom_version.sh tool generates a content bundle containing a bootstrap and cluster profile for a desired version of openshift.

OS_VERSION=latest-4.16
ops-tools/bundle_custom_version.sh $OS_VERSION >openshift-$OS_VERSION.yaml
drpcli contents upload openshift-$OS_VERSION.yaml

The bootstrap-openshift-<version> profile can be added to the self-runner to download the needed artifacts when running the rebootstrap-drp blueprint or the universal-bootstrap workflow.

The openshift-cluster-<version> profile can be added during cluster creation to specify which version of openshift and coreos to install.

Airgap

All required files can be downloaded using the download_airgap_artifacts.sh tool. This tool also provides the DRPCLI file and iso commands needed to upload the required files.

ops-tools/download_airgap_artifacts.sh 4.15.46

Support

For issues or questions: - Check the Digital Rebar documentation - Review the OpenShift documentation - Review the troubleshooting section - Contact RackN support

License

RackN License - See documentation for details.

.. Release v4.15.0 Start