23.30. kube-lib - Kubernetes Library¶

The following documentation is for Kubernetes Library (kube-lib) content package at version v4.9.0-alpha00.67+g2af61053ec23a91be2e566ecf386ec23b2ae1119.

License: Kube-Lib is APLv2

This document provides information on how to use the Digital Rebar Kube-Lib content add-on. Use of this content supports common functions needed for other Kubernetes workflows like KRIB and EdgeLab.

23.30.1. Digital Rebar Kube-Lib¶

Kube-Lib is a library of standard Kubernetes (and k3s) installation processes leveraging v4.6 cluster pattern and other advanced features of Digital Rebar.

Note: Kube-Lib replaces install components in KRIB and edge-lab.

23.30.1.1. Local vs Online Requirements¶

By default, Kube-Lib uses online components; it will attempt to download and store local copies when posssible. For that reason, our goal is to look for local copies first. Depending on the use-case, that may allow operators to use Kube-Lib is Air-Gap environments by pre-populating resources.

23.30.1.2. Kube-Lib Basics¶

Kube-Lib is a Content Pack addition to Digital Rebar Provision. It uses the rs_cluster_pattern which provides atomic guarantees. This allows for Kubernetes leader(s) to be dynamically elected, forcing all other nodes to wait until the kubeadm on the elected leader to generate an installation token for the rest of the nodes. Once the Kubernetes leader is bootstrapped, the Digital Rebar system facilitates the security token hand-off to rest of the cluster so they can join without any operator intervention.

23.30.1.3. Elected -vs- Specified Leaders¶

By default, the Kube-Lib process will dynamically elect a leader(s) for the Kubernetes cluster. This leader is simply selected by the cluster manager and the rest of the cluster will coalesce around the elected leader(s).

If you wish to specify a specific machines to be the designated leaders using cluster/leader to true, you can do so by setting a Param on the target machine(s).

23.30.2. Operating Kube-Lib¶

23.30.2.1. Use `kubectl` - from anywhere¶

Once the Kubernetes cluster build has been completed, you may use the kubectl command to both verify and manage the cluster. You will need to download the conf file with the appropriate tokens and information to connect to and authenticate your kubectl connections. Below is an example of doing this:

# get the Admin configuration and tokens
drpcli profiles get cluster01 param kube-lib/kube-conf --expand > kube.conf
export KUBECONFIG=~/Downloads/kube.conf
kubectl get nodes

23.30.3. Object Specific Documentation¶

23.30.3.1. params¶

The content package provides the following params.

23.30.3.1.1. kube-lib/api-port¶

The API bindPort number for the kubernetes/k3s cluster . Defaults to ‘6443’.

23.30.3.1.2. kube-lib/dashboard-token¶

Param is set (output) by the Dashboard install process

To start the dashboard, use kubectl proxy then open http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

Note: this is stored as a secure object so the decode is required

23.30.3.1.3. kube-lib/dashboard-version¶

Version of Dashboard to install.

Note: Required “v” prefix or task Will skip Dashboard install.

Override the default in cluster or global profile to use a different version.

23.30.3.1.4. kube-lib/helm-charts¶

23.30.3.1.5. Install Helm Charts¶

Array of charts to install via Helm. The list will be followed in order.

Work is idempotent: No action is taken if charts are already installed.

Fields: chart and name are required.

Options exist to inject additional control flags into helm install instructions:

name: name of the chart (required)
chart: reference of the chart (required) - may rely on repo, path or other helm install [chart] standard
namespace: kubernetes namespace to use for chart (defaults to none)
params: map of parameters to include in the helm install (optional). Keys and values are converted to –[key] [value] in the install instruction.
set: array of values to set in the helm install (optional). values are converted to –set [value] in the install instruction.
targz (optional) provides a location for a tar.gz file containing charts to install. Path is relative.
repos (optional) adds the requested repos to helm using helm repo add before installing helm. syntax is [repo name]: [repo path].
sleep (optional): time in seconds to wait after install (defaults to none)
postkubectl (optional) map of kubectl [request] commands to run after the helm install - stored using the map key
prekubectl (optional) array of kubectl [request] commands to run before the helm install
git (optional): array of urls for cloning repos into local path

NOT YET PORTED FROM KRIB * wait: wait for name (and namespace if provided) to be running before next action * templates (optional) map of DRP templates keyed to the desired names (must be uploaded!) to render before doing other work. * templatesbefore (optional) expands the provided template files inline before the helm install happens. * templatesafter (optional) expands the provided template files inline after the helm install happens

example:

[
  {
    "chart": "stable/mysql",
    "name": "mysql"
  }, {
    "chart": "openfaas/openfaas",
    "name": "openfaas",
    "namespace": "openfaas",
    "git": [
      "https://github.com/openfaas/faas-netes/"
    ],
    "repos": {
      "openfaas":"https://openfaas.github.io/faas-netes/"
      },
    "prekubectl": [
       "apply -f ./faas-netes/namespaces.yml",
       "apply -f ./faas-netes/yaml_amd64"
    ],
    "postkubectl": {
      "openfaas/password": "-n openfaas get secret basic-auth -o jsonpath=\"{.data.basic-auth-password}\" | base64 --decode"
    },
    "set": [
      "functionNamespace=openfaas-fn",
      "generateBasicAuth=true"
    ],
    "sleep": 10
  }
]

23.30.3.1.6. kube-lib/helm-version¶

Allows operators to determine the version of Helm to install Includes the download URLs and sha256sums Uses task-lib download-utilities format

23.30.3.1.7. kube-lib/k3s-version¶

Allows operators to determine the version of k3s to install Includes the download URLs and sha256sums

23.30.3.1.8. kube-lib/kube-config¶

Param is set (output) by the cluster building process

To use the file, use one of the following * save to $HOME/.kube * export KUBECONFIG=[file] * pass –kubeconfig=[kube.config] to the kubectl or other tools.

If your cluster is using the default cluster01 then tou can retrieve the kube.config file using: drpcli profiles get cluster01 param kube-lib/kube-config –decode > kube.config

Note: this is stored as a secure object so the decode is required

23.30.3.1.9. kube-lib/kubectl-version¶

Allows operators to determine the version of kubectl to install Includes the download URLs and sha256sums

23.30.3.1.10. kube-lib/kubernetes-version¶

Allows operators to specify the version of Kubernetes to install

to get latest, curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt

23.30.3.1.11. kube-lib/node-token¶

Param is set (output) by the leader during cluster building process HAS NO DEFAULT: param is used as a timing gate between the Leader and Workers

23.30.3.1.12. openfaas/password¶

Param is set (output) by the leader during OpenFaaS helm chart install.

23.30.3.2. profiles¶

The content package provides the following profiles.

23.30.3.2.1. helm-openfaas¶

Overrides Array of Helm charts to install OpenFaaS

23.30.3.3. stages¶

The content package provides the following stages.

23.30.3.3.1. kube-lib-dashboard¶

Installs and runs Kubernetes Dashboard after a cluster has been constructed. This stage is idempotent and can be run multiple times.

Note: will skip if the dashboard version does not start with v

23.30.3.3.2. kube-lib-helm¶

Installs and runs Helm Charts after a cluster has been constructed.

This stage is idempotent and can be run multiple times. This allows operators to create workflows with multiple instances of this stage. The charts to run are determined by the edge-lab/helm-charts parameter.

Unless helm is uploaded to file helm/helm, this stage requires internet access.

23.30.3.3.3. kube-lib-k3s-cluster¶

Coordinates operations across multiple machines to install k3s using the v4.6 cluster manager patter

Downloads k3s to cache
Uses kube-lib-k3s-leader to install the leader
Uses kube-lib-k3s-worker to install the workers

23.30.3.3.4. kube-lib-k3s-machine-install¶

Installs k3s using v4.6 cluster pattern Will install leader if cluster/leader: true Will install worker if cluster/leader: false

23.30.3.4. tasks¶

The content package provides the following tasks.

23.30.3.4.1. kube-lib-dashboard-install¶

Installs Dashboard on the Cluster. Saves install token into k3s/dashboard-token Note: will skip if the dashboard version does not start with v

To start the dashboard, use kubectl proxy then open http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

23.30.3.4.2. kube-lib-helm-charts¶

Runs Helm based on Charts defined in kube-lib/helm-carts.

There are a wide range of options available that are defined in the Param

This uses the Digital Rebar Cluster pattern so cluster/profile and cluster/leader must be set.

The task is designed to be idempotent; however, some helm operations are not.

23.30.3.4.3. kube-lib-helm-install¶

Installs Helm on the leader. This uses the Digital Rebar Cluster pattern so cluster/profile and cluster/leader must be set.

23.30.3.4.4. kube-lib-k3s-cluster¶

This uses the v4.6+ cluster install pattern so it MUST be started from the cluster manager, not from one of the members of the cluster.

If the cluster/filter does not match any machines then the task fail and allow operators to create the correct selectors.

For operational guidelines, see rs_cluster_pattern

23.30.3.4.5. kube-lib-k3s-machine-install¶

Install k3s on machines based on v4.6 cluster pattern. Generally, this is started by the kube-lib-k3s-cluster workflow and not called directly.

cluster/leader: true becomes the cluster leader. others become workers. cluster/manager: no operation

For operational guidelines, see rs_cluster_pattern

23.30.3.4.6. kube-lib-kubectl-install¶

Installs KubeCTL on machine.

Looks up from ‘kube-lib/kubernetes-download-url’ using ‘kube-lib/kubernetes-version’

Saves binary to files/kubernetes/kubectl-[arch] so only download is required.

Air-Gap: populate files/kubernetes/kubectl-[arch] to avoid download.

23.30.3.5. workflows¶

The content package provides the following workflows.

23.30.3.5.1. k3s-cluster-build¶

Builds a k3s cluster and secondary actions on all machines in lab Uses the task-library cluster building components

Downloads from the Rancher Github (k3s/download-url) unless binary is already present as /files/k3s/k3s

23.30.3.5.2. k3s-machine-install¶

Installs k3s on a machine (can be either worker or leader) Designed to be run by k3s-cluster-builder

23.30. kube-lib - Kubernetes Library¶

23.30.1. Digital Rebar Kube-Lib¶

23.30.1.1. Local vs Online Requirements¶

23.30.1.2. Kube-Lib Basics¶

23.30.1.3. Elected -vs- Specified Leaders¶

23.30.2. Operating Kube-Lib¶

23.30.2.1. Use kubectl - from anywhere¶

23.30.3. Object Specific Documentation¶

23.30.3.1. params¶

23.30.3.1.1. kube-lib/api-port¶

23.30.3.1.2. kube-lib/dashboard-token¶

23.30.3.1.3. kube-lib/dashboard-version¶

23.30.3.1.4. kube-lib/helm-charts¶

23.30.3.1.5. Install Helm Charts¶

23.30.3.1.6. kube-lib/helm-version¶

23.30.3.1.7. kube-lib/k3s-version¶

23.30.3.1.8. kube-lib/kube-config¶

23.30.3.1.9. kube-lib/kubectl-version¶

23.30.3.1.10. kube-lib/kubernetes-version¶

23.30.3.1.11. kube-lib/node-token¶

23.30.3.1.12. openfaas/password¶

23.30.3.2. profiles¶

23.30.3.2.1. helm-openfaas¶

23.30.3.3. stages¶

23.30.3.3.1. kube-lib-dashboard¶

23.30.3.3.2. kube-lib-helm¶

23.30.3.3.3. kube-lib-k3s-cluster¶

23.30.3.3.4. kube-lib-k3s-machine-install¶

23.30.3.4. tasks¶

23.30.3.4.1. kube-lib-dashboard-install¶

23.30.3.4.2. kube-lib-helm-charts¶

23.30.3.4.3. kube-lib-helm-install¶

23.30.3.4.4. kube-lib-k3s-cluster¶

23.30.3.4.5. kube-lib-k3s-machine-install¶

23.30.3.4.6. kube-lib-kubectl-install¶

23.30.3.5. workflows¶

23.30.3.5.1. k3s-cluster-build¶

23.30.3.5.2. k3s-machine-install¶

23.30.2.1. Use `kubectl` - from anywhere¶