22.32. kube-lib - Kubernetes Library

The following documentation is for Kubernetes Library (kube-lib) content package at version v4.11.0-alpha00.69+g6fdc0a93b761e2a35c704a1d9fae2f0cd104db20.

License: Kube-Lib is APLv2

This document provides information on how to use the Digital Rebar Kube-Lib content add-on. Use of this content supports common functions needed for other Kubernetes workflows like KRIB and EdgeLab.

22.32.1. Digital Rebar Kube-Lib

Kube-Lib is a library of standard Kubernetes (and k3s) installation processes leveraging v4.6 cluster pattern and other advanced features of Digital Rebar.

Note: Kube-Lib replaces install components in KRIB and edge-lab.

22.32.1.1. Local vs Online Requirements

By default, Kube-Lib uses online components; it will attempt to download and store local copies when posssible. For that reason, our goal is to look for local copies first. Depending on the use-case, that may allow operators to use Kube-Lib is Air-Gap environments by pre-populating resources.

22.32.1.2. Kube-Lib Basics

Kube-Lib is a Content Pack addition to Digital Rebar Provision. It uses the Clusters v4.8+ which provides atomic guarantees. This allows for Kubernetes leader(s) to be dynamically elected, forcing all other nodes to wait until the kubeadm on the elected leader to generate an installation token for the rest of the nodes. Once the Kubernetes leader is bootstrapped, the Digital Rebar system facilitates the security token hand-off to rest of the cluster so they can join without any operator intervention.

22.32.1.3. Elected -vs- Specified Leaders

By default, the Kube-Lib process will dynamically elect a leader(s) for the Kubernetes cluster. This leader is simply selected by the cluster manager and the rest of the cluster will coalesce around the elected leader(s).

If you wish to specify a specific machines to be the designated leaders using cluster/leader to true, you can do so by setting a Param on the target machine(s).

22.32.2. Operating Kube-Lib

22.32.2.1. Use kubectl - from anywhere

Once the Kubernetes cluster build has been completed, you may use the kubectl command to both verify and manage the cluster. You will need to download the conf file with the appropriate tokens and information to connect to and authenticate your kubectl connections. Below is an example of doing this:

# get the Admin configuration and tokens
drpcli profiles get cluster01 param kube-lib/kube-conf --expand > kube.conf
export KUBECONFIG=~/Downloads/kube.conf
kubectl get nodes

22.32.3. Object Specific Documentation

22.32.3.1. blueprints

The content package provides the following blueprints.

22.32.3.1.1. universal-application-kubernetes-cluster

WorkOrder maintenance for Kubernetes Cluster

This is used when a machine is in WorkOrder mode and needs to maintain the pipeline

22.32.3.1.2. universal-application-kubernetes-machine

WorkOrder maintenance for Kubernetes Machines

This is used when a machine is in WorkOrder mode and needs to maintain the pipeline

22.32.3.2. params

The content package provides the following params.

22.32.3.2.1. kube-lib/api-port

The API bindPort number for the kubernetes/k3s cluster . Defaults to ‘6443’.

22.32.3.2.2. kube-lib/cni-plugins-version

Allows operators to specify the version of CNI plugins to install

22.32.3.2.3. kube-lib/containerd-version

Allows operators to specify the version of ContainerD to install

22.32.3.2.4. kube-lib/crictl-version

Allows operators to specify the version of CRIctl to install

22.32.3.2.5. kube-lib/dashboard-token

Param is set (output) by the Dashboard install process

To start the dashboard, use kubectl proxy then open http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

Note: this is stored as a secure object so the decode is required

22.32.3.2.6. kube-lib/dashboard-version

Version of Dashboard to install.

Note: Required “v” prefix or task Will skip Dashboard install.

Override the default in cluster or global profile to use a different version.

22.32.3.2.7. kube-lib/helm-charts

22.32.3.2.8. Install Helm Charts

Array of charts to install via Helm. The list will be followed in order.

Work is idempotent: No action is taken if charts are already installed.

Fields: chart and name are required.

Options exist to inject additional control flags into helm install instructions:

  • name: name of the chart (required)

  • chart: reference of the chart (required) - may rely on repo, path or other helm install [chart] standard

  • namespace: kubernetes namespace to use for chart (defaults to none)

  • params: map of parameters to include in the helm install (optional). Keys and values are converted to –[key] [value] in the install instruction.

  • set: array of values to set in the helm install (optional). values are converted to –set [value] in the install instruction.

  • targz (optional) provides a location for a tar.gz file containing charts to install. Path is relative.

  • repos (optional) adds the requested repos to helm using helm repo add before installing helm. syntax is [repo name]: [repo path].

  • sleep (optional): time in seconds to wait after install (defaults to none)

  • postkubectl (optional) map of kubectl [request] commands to run after the helm install - stored using the map key

  • prekubectl (optional) array of kubectl [request] commands to run before the helm install

  • git (optional): array of urls for cloning repos into local path

NOT YET PORTED FROM KRIB * wait: wait for name (and namespace if provided) to be running before next action * templates (optional) map of DRP templates keyed to the desired names (must be uploaded!) to render before doing other work. * templatesbefore (optional) expands the provided template files inline before the helm install happens. * templatesafter (optional) expands the provided template files inline after the helm install happens

example:

[
  {
    "chart": "stable/mysql",
    "name": "mysql"
  }, {
    "chart": "openfaas/openfaas",
    "name": "openfaas",
    "namespace": "openfaas",
    "git": [
      "https://github.com/openfaas/faas-netes/"
    ],
    "repos": {
      "openfaas":"https://openfaas.github.io/faas-netes/"
      },
    "prekubectl": [
       "apply -f ./faas-netes/namespaces.yml",
       "apply -f ./faas-netes/yaml_amd64"
    ],
    "postkubectl": {
      "openfaas/password": "-n openfaas get secret basic-auth -o jsonpath=\"{.data.basic-auth-password}\" | base64 --decode"
    },
    "set": [
      "functionNamespace=openfaas-fn",
      "generateBasicAuth=true"
    ],
    "sleep": 10
  }
]

22.32.3.2.9. kube-lib/helm-version

Allows operators to determine the version of Helm to install Includes the download URLs and sha256sums Uses task-lib download-utilities format

22.32.3.2.10. kube-lib/kube-config

Param is set (output) by the cluster building process

To use the file, use one of the following:

  • save to $HOME/.kube

  • export KUBECONFIG=[file]

  • pass –kubeconfig=[kube.config] to the kubectl or other tools.

If your cluster is using the default cluster01 then tou can retrieve the kube.config file using: drpcli profiles get cluster01 param kube-lib/kube-config –decode > kube.config

Note: this is stored as a secure object so the decode is required

22.32.3.2.11. kube-lib/kubeadm-version

Allows operators to specify the version of Kubeadm to install

to get latest, curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt

22.32.3.2.12. kube-lib/kubectl-version

Allows operators to determine the version of kubectl to install Includes the download URLs and sha256sums

22.32.3.2.13. kube-lib/kubelet-version

Allows operators to specify the version of Kubelet to install

22.32.3.2.14. kube-lib/node-token

Param is set (output) by the leader during cluster building process HAS NO DEFAULT: param is used as a timing gate between the Leader and Workers

22.32.3.2.15. kube-lib/role

Role determines what is installed on a machine in the cluster

Defined roles are:

  • control: provides all the control functions (api & etcd)

  • node: worker functions (kubelet)

22.32.3.2.16. kube-lib/secrets

Keys, Certificates and Configuration files generated from kubeadm init process and used to bring up leaders and workers.

Keys are file locations. Individual values are base64 encoded

22.32.3.3. stages

The content package provides the following stages.

22.32.3.3.1. kube-lib-helm

Installs and runs Helm Charts after a cluster has been constructed.

This stage is idempotent and can be run multiple times. This allows operators to create workflows with multiple instances of this stage. The charts to run are determined by the edge-lab/helm-charts parameter.

Unless helm is uploaded to file helm/helm, this stage requires internet access.

22.32.3.4. tasks

The content package provides the following tasks.

22.32.3.4.1. kube-lib-containerd-install

Installs containerd cni from the binary version

NO TUNING PROVIDED AT THIS TIME.

Ref: https://github.com/containerd/containerd/

22.32.3.4.2. kube-lib-dashboard-install

Installs Kubernetes Dashboard on the Cluster.

Prereq: kubectl must be installed (use kube-lib-kubectl-install task) before running this task

OPTION: will skip if the dashboard version does not start with v

To start the dashboard, use kubectl proxy then open http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

22.32.3.4.3. kube-lib-helm-charts

Runs Helm based on Charts defined in kube-lib/helm-carts.

There are a wide range of options available that are defined in the Param

This uses the Digital Rebar Cluster pattern so cluster/profile and cluster/leader must be set.

The task is designed to be idempotent; however, some helm operations are not.

22.32.3.4.4. kube-lib-helm-install

Installs Helm on the leader. This uses the Digital Rebar Cluster pattern so cluster/profile and cluster/leader must be set.

22.32.3.4.5. kube-lib-kubeadm-initialize

Setups Kubeadm on cluster.

22.32.3.4.6. kube-lib-kubeadm-install

Installs Kubeadm on machine.

22.32.3.4.7. kube-lib-kubeadm-machine

Uses Kubeadm to build or update a k8s cluster based on the kube-lib/role assignment.

No action if kube-lib/secrets has not been defined by kube-lib-kubeadm-initialize.

22.32.3.4.8. kube-lib-kubectl-install

Installs KubeCTL on machine.

22.32.3.4.9. kube-lib-kubelet-install

Installs Kubelet on machine.

Saves binary to files/kubernetes/kubectl-[arch] so only download is required.

Air-Gap: populate files/kubernetes/kubectl-[arch] to avoid download.