Backstage Integration

This guide will guide you in setting up basic Backstage integration with Digital Rebar.

You will be creating a simple backend plugin to provide functionality to the template and frontend plugins you will create afterward. The end goal is a simple integration that will allow you to quickly spin up, view, and destroy Clusters in DRP.

It is composed of three parts:

The backend plugin, a requirement for the following two parts,
The scaffolder, a sample template to spin up DRP Clusters, and
The frontend plugin, a view for listing Clusters and deleting them.

Backend¶

The backend plugin is responsible for providing custom actions to the scaffolder and API functionality to the frontend.

In this tutorial, we will be setting up the aforementioned plugin to work with our two other components.

We will:

Create the plugin
Optionally add custom actions for the scaffolder
Optionally add router extensions for the frontend
Export our plugin changes
Authorize our Backstage instance with DRP
Register the plugin in Backstage

While you can choose to omit adding custom actions or router extensions, it is recommended that you do both so that you get a full sense for Digital Rebar's power with Backstage.

Info

A sample, reference backend plugin is available at https://gitlab.com/zfranks/backstage-plugin-digitalrebar-backend.

Creating the plugin¶

The official documentation for creating a backend plugin is available here, but we will walk through it for our cases.

Start from the root directory of your Backstage instance.

yarn new --select backend-plugin

Let's name (ID) our plugin digitalrebar. This will create a package at plugins/digitalrebar-backend. The package will be named @internal/plugin-digitalrebar-backend. You are free to rename these after this tutorial.

Using the Digital Rebar TypeScript API¶

From the root of your Backstage instance, issue the following command to add the DRP TS API to your plugin.

yarn add --cwd plugins/digitalrebar-backend @rackn/digitalrebar-api

This will add the official Digital Rebar TypeScript API to your backend plugin. It can be used as a lightweight wrapper for DRP REST API calls, which will greatly simplify the implementation of our custom actions and the functionality our frontend should present to the user.

Adding custom actions¶

First, open up your Backstage instance in the text editor of your choice. In your plugin directory at plugins/digitalrebar-backend, create a new directory actions:

mkdir plugins/digitalrebar-backend/actions

Create a new file clusters.ts in this folder. It will hold the functionality for our drp:clusters:create action we will use in the scaffolder.

Set clusters.ts to the following. We will break it down after the source.

import { Config } from '@backstage/config';
import type { JsonValue } from '@backstage/types';
import { createTemplateAction } from '@backstage/plugin-scaffolder-backend';
import DRApi, { DRMachine } from '@rackn/digitalrebar-api';

export const clustersActions = (config: Config) => {
  // DRP endpoints use self-signed certificates
  // you may want to set this manually, but it is here
  // for development purposes
  process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0';

  return [
    createTemplateAction({
      id: 'drp:clusters:create',
      schema: {
        input: {
          type: 'object',
        },
      },
      handler: async (ctx) => {
        const endpoint = config.getString('digitalrebar.endpoint');
        const token = config.getString('digitalrebar.token');
        const api = new DRApi(endpoint);
        api.setToken(token);

        ctx.logger.info('Creating cluster...');

        const response = await api.clusters.create(
          ctx.input as unknown as DRMachine
        );

        ctx.logger.info(`Created cluster with UUID ${response.data.Uuid}`);
        ctx.output('object', response.data as unknown as JsonValue);
        ctx.output('endpoint', endpoint);
      },
    }),
  ];
};

Ignoring our import section, let's look at the entire thing, broken down:

export const clustersActions = (config: Config) => {
  // DRP endpoints use self-signed certificates
  // you may want to set this manually, but it is here
  // for development purposes
  process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0';

This section sets the NODE_TLS_REJECT_UNAUTHORIZED environment variable to 0, as DRP endpoints use self-signed certificates. You may want to set this elsewhere, or remove it entirely if your DRP endpoint is properly signed.

  return [
    createTemplateAction({
      id: 'drp:clusters:create',
      schema: {
        input: {
          type: 'object',
        },
      },

We return an array of actions. In this case, we are only creating one action. Actions are created using the Backstage-provided createTemplateAction. We will call it drp:clusters:create. Its input schema is simply an object with no known parameters; it simply gets shipped to our DRP API as the object we are trying to create. In this way, the input object of this action can be an entire DRP Cluster.

      handler: async (ctx) => {
        const endpoint = config.getString('digitalrebar.endpoint');
        const token = config.getString('digitalrebar.token');
        const api = new DRApi(endpoint);
        api.setToken(token);

In this section, we grab the endpoint and token config options from our app-config.yaml (see the section on DRP authorization) and we give them to our DRApi so that we can make calls to our DRP endpoint.

        ctx.logger.info('Creating cluster...');

        const response = await api.clusters.create(
          ctx.input as unknown as DRMachine
        );

Here, we note to the user that we are creating the cluster. Then, we call api.clusters.create with our input object.

Note

Note the TypeScript that is happening here (as unknown as DRMachine). Backstage provides input as a JsonValue, which cannot be properly cast to DRMachine. We trust the conversion, so we assume input is unknown, and then DRMachine, so that the value can be passed into the create method.

        ctx.logger.info(`Created cluster with UUID ${response.data.Uuid}`);
        ctx.output('object', response.data as unknown as JsonValue);
        ctx.output('endpoint', endpoint);

We notify the user that the cluster has been created, and its UUID is fetched from response.data.Uuid. The entire Cluster object that was created is available at response.data, for that matter.

We set our outputs:

object is our response.data described above. Note again the TypeScript casting that is used for the inverse case.
endpoint is our endpoint we got from our configuration a few sections above. Again, see the section on DRP authorization.

Adding router extensions¶

In order to provide functionality to the frontend, we need to extend Backstage's router in our backend plugin.

Create a new folder service in your backend plugin's directory if it does not exist:

mkdir plugins/digitalrebar-backend/service

Then, create a file router.ts in it and set it to the following. Again, we will break it down afterwards.

There is a chance this file already exists for you in your boilerplate plugin. If it does, make sure to add the relevant parts discussed below.

import { errorHandler } from '@backstage/backend-common';
import { Config } from '@backstage/config';
import DRApi, { DRWorkOrder } from '@rackn/digitalrebar-api';
import express from 'express';
import Router from 'express-promise-router';
import { Logger } from 'winston';

export interface RouterOptions {
  logger: Logger;
  config: Config;
}

export async function createRouter(
  options: RouterOptions
): Promise<express.Router> {
  const { logger } = options;

  const router = Router();
  router.use(express.json());

  const endpoint = options.config.getString('digitalrebar.endpoint');
  const token = options.config.getString('digitalrebar.token');

  const api = new DRApi(endpoint);
  api.setToken(token);

  router.get('/health', (_, response) => {
    logger.info('PONG!');
    response.json({ status: 'ok' });
  });

  router.get('/clusters', async (_, response) => {
    response.json((await api.clusters.list({ aggregate: 'true' })).data);
  });

  router.delete('/clusters/:uuid', async (request, response) => {
    response.json((await api.clusters.delete(request.params.uuid)).data);
  });

  router.use(errorHandler());
  return router;
}

Much of this is Backstage boilerplate, and is somewhat explained in the official documentation. The parts we care about are below:

  const endpoint = options.config.getString('digitalrebar.endpoint');
  const token = options.config.getString('digitalrebar.token');

  const api = new DRApi(endpoint);
  api.setToken(token);

This should look familiar if you completed the custom actions part of this tutorial.

  router.get('/clusters', async (_, response) => {
    response.json((await api.clusters.list({ aggregate: 'true' })).data);
  });

This call adds a listener to GET /clusters to our router. It returns a list of Clusters from our DRP endpoint, thanks to the api.clusters.list method provided by the DRP TypeScript API. We also set aggregate: 'true' here so that DRP responds with all Params, even those across inherited Profiles, as all Clusters have an associated Profile.

  router.delete('/clusters/:uuid', async (request, response) => {
    response.json((await api.clusters.delete(request.params.uuid)).data);
  });

Like the call above, this will add a listener to DELETE /clusters/:uuid. It will be responsible for deleting clusters given their UUID.

Our plugin is now prepared to listen to API calls from our frontend.

Exporting our plugin functionality¶

To make our backend functionality visible to the rest of Backstage, set your plugin's index.ts to the following:

// Export our cluster actions
export * from './actions/clusters.ts';

// Export our router extensions for the frontend
export * from './service/router.ts';

DRP Authorization¶

In order for your Backstage instance to be able to make requests to DRP, you need to add some information to your instance's app-config.yaml.

First, you'll need to get your endpoint. Your endpoint should be the IP address followed by the port DRP listens to (by default, 8092). It is the same as the IP and port you use to log into the DRP UX. You'll replace YOUR-ENDPOINT in the following yaml excerpt with this address.

Then, you'll need a DRP user's token that the plugin will make requests through. Log in to a user on your DRP instance through the portal, and click Copy Auth Token at the very bottom of the left sidebar. You'll replace YOUR-TOKEN in the following with this.

Open app-config.yaml in the text editor of your choice, and add the following at the top-level somewhere in the file:

digitalrebar:
  endpoint: YOUR-ENDPOINT
  token: YOUR-TOKEN

Register the plugin¶

To register the plugin with Backstage, you'll have to create and modify a few files.

If you made router extensions¶

Start by creating packages/backend/src/plugins/digitalrebar.ts and setting it to the following.

import { createRouter } from '@internal/plugin-digitalrebar-backend';
import { Router } from 'express';
import { PluginEnvironment } from '../types';

export default async function createPlugin(
  env: PluginEnvironment
): Promise<Router> {
  return await createRouter({ logger: env.logger, config: env.config });
}

Then, open packages/backend/src/index.ts, and add:

// imports ...
import digitalrebar from './plugins/digitalrebar'; // add this line

// down a bit, in the main function ...
async function main() {
  // ... more environments here ...
  const appEnv = useHotMemoize(module, () => createEnv('app'));
  const drpEnv = useHotMemoize(module, () => createEnv('drp')); // add this line

  // ...
  apiRouter.use('/search', await search(searchEnv));
  apiRouter.use('/drp', await digitalrebar(drpEnv)); // add this line

  // ...
}

This will register the plugin router with Backstage's API router.

If you added custom actions¶

Add the following to packages/backend/src/plugins/scaffolder.ts:

import { clustersActions } from '@internal/plugin-digitalrebar-backend';

export default async function createPlugin(
  env: PluginEnvironment
): Promise<Router> {
  const catalogClient = new CatalogClient({
    discoveryApi: env.discovery,
  });

  const integrations = ScmIntegrations.fromConfig(env.config);

  // since we are adding actions, we must manually create the built-in actions
  // if you are already adding custom actions, simply add the `...clustersActions` line
  const actions = [
    ...createBuiltinActions({
      catalogClient,
      integrations,
      config: env.config,
      reader: env.reader,
    }),
    // add this following line
    ...clustersActions(env.config),
  ];

  return await createRouter({
    logger: env.logger,
    config: env.config,
    database: env.database,
    reader: env.reader,
    catalogClient,
    identity: env.identity,
    actions, // add this line if you were not using custom actions before
  });
}

Usage¶

As an independent component, the backend plugin does not do much. That said, continue on to the scaffolder (if you added custom actions) or to the frontend (if you added router extensions). If you did both, choose either!

Frontend¶

The frontend plugin is used to display information in the Backstage portal, and uses the backend plugin to communicate with Digital Rebar.

In this tutorial, we will create and set up the frontend plugin so that we can view a list of Clusters in our DRP endpoint, as well as a way to delete clusters.

If you have not yet completed the backend tutorial, please complete that first, as there is functionality required by the frontend!

This tutorial assumes a basic understanding of React.

Info

A sample, reference frontend plugin is available at https://gitlab.com/zfranks/backstage-plugin-digitalrebar.

Creating the plugin¶

The official documentation for creating a frontend plugin is available here, but we will walk through it for our cases.

Start from the root directory of your Backstage instance.

yarn new --select plugin

Use the same ID as our backend plugin, digitalrebar.

This will create a package at plugins/digitalrebar. The package will be named @internal/plugin-digitalrebar. You are free to rename these after this tutorial.

Adding the plugin page¶

Let's start by adding some components.

Create plugins/digitalrebar/src/components/Clusters/View.tsx with the following source:

import React from 'react';
import { Typography, Grid } from '@material-ui/core';
import {
  InfoCard,
  Header,
  Page,
  Content,
  ContentHeader,
  HeaderLabel,
  SupportButton,
} from '@backstage/core-components';
import { ClustersTable } from './Table';

export const ClustersView = () => (
  <Page themeId="tool">
    <Header title="Welcome to Digital Rebar!">
      <HeaderLabel label="Owner" value="RackN" />
      <HeaderLabel label="Lifecycle" value="Alpha" />
    </Header>
    <Content>
      <ContentHeader title="Digital Rebar">
        <SupportButton>Some description goes here.</SupportButton>
      </ContentHeader>
      <Grid container spacing={3} direction="column">
        <Grid item>
          <ClustersTable />
        </Grid>
      </Grid>
    </Content>
  </Page>
);

This is just React filler that follows Backstage's conventions for plugin pages. It should be reminiscent of the included boilerplate ExampleComponent.

Now, create plugins/digitalrebar/src/components/Clusters/Table.tsx with the following source:

import { Progress, Table, TableColumn } from '@backstage/core-components';
import { configApiRef, errorApiRef, useApi } from '@backstage/core-plugin-api';
import Alert from '@material-ui/lab/Alert';
import { DRMachine } from '@rackn/digitalrebar-api';
import React, { useCallback, useMemo, useState } from 'react';
import useAsync from 'react-use/lib/useAsync';
import DeleteIcon from '@material-ui/icons/Delete';
import AddIcon from '@material-ui/icons/Add';
import RemoveIcon from '@material-ui/icons/Remove';
import { IconButton, Link } from '@material-ui/core';

const ClusterActions: React.FC<{ data: DRMachine; type: 'row' | 'group' }> = ({
  data,
}) => {
  const config = useApi(configApiRef);
  const [loading, setLoading] = useState(false);
  const [deleted, setDeleted] = useState(false);

  const onDelete = useCallback(() => {
    if (loading) return;
    setLoading(true);

    fetch(
      `${config.getString('backend.baseUrl')}/api/drp/clusters/${data.Uuid}`,
      { method: 'DELETE' }
    )
      .then(() => {
        setDeleted(true);
      })
      .finally(() => {
        setLoading(false);
      });
  }, [config, loading, data.Uuid]);

  if (deleted) return <i>Deleted.</i>;
  if (loading) return <Progress />;

  return (
    <IconButton size="small" onClick={onDelete}>
      <DeleteIcon />
    </IconButton>
  );
};

export const ClustersTable = () => {
  const config = useApi(configApiRef);
  const endpoint = useMemo(
    () => config.getString('digitalrebar.endpoint'),
    [config]
  );

  const { value, loading, error } = useAsync(async (): Promise<DRMachine[]> => {
    return await fetch(
      `${config.getString('backend.baseUrl')}/api/drp/clusters`
    ).then((r) => r.json());
  }, []);

  const columns = useMemo<TableColumn<DRMachine>[]>(
    () => [
      {
        title: 'Name',
        render: (data, _) => (
          <Link
            href={`https://portal.rackn.io/#/e/${endpoint}/clusters/${data.Uuid}`}
          >
            {data.Name}
          </Link>
        ),
      },
      { title: 'Size', field: 'Params.cluster/count' },
      { title: 'Broker', field: 'Params.broker/name' },
      {
        title: 'Actions',
        render: (data, type) => <ClusterActions data={data} type={type} />,
      },
    ],
    [endpoint]
  );

  if (loading) {
    return <Progress />;
  } else if (error) {
    return <Alert severity="error">{error.message}</Alert>;
  }

  return (
    <Table
      title="Clusters"
      options={{ search: false, paging: false }}
      columns={columns}
      data={value ?? []}
    />
  );
};

Let's break down each of the components individually.

`ClusterActions`¶

This component provides the delete button, allowing us to delete Clusters from the table. Of its source, most notably is the call to /api/drp/clusters/:uuid, which we added in our backend plugin.

`ClustersTable`¶

A table of Clusters. Includes some basic information about each cluster, as well as the ClusterActions component allowing us to remove clusters by their row.

useApi and useAsync are hooks provided by Backstage, as well as the Table component (and its child TableColumns). There is an official Backstage tutorial that goes into further detail on these pieces.

You'll notice the call to useAsync uses the GET /clusters route we added in our backend plugin. The preceding /drp was declared when we registered the backend plugin with Backstage.

Registering the plugin with Backstage¶

Conveniently, creating the plugin automatically registers it within Backstage's frontend router, so there are no steps that must be taken from the perspective of this tutorial.

Register it in another instance¶

If you would like to register it with another Backstage instance after publishing the plugin, check the diffs in packages/app/src/App.tsx after creating the plugin. It should be the addition of an import line and a new Route entry.

Usage¶

Start your Backstage instance with

yarn dev

and navigate to http://localhost:3000/digitalrebar to see your changes.

Optionally, create a Cluster with the template you set up in the scaffolder tutorial, and report back to see if it is now in the frontend table. Check to see that the delete icon deletes your Cluster.

Scaffolder¶

The scaffolder, otherwise known as software templates by the Backstage docs, is a way to create objects based on defined templates. The sample template that ships with a standard Backstage installation is a Node.js app.

In the case of Digital Rebar, templates can be used to quickly spin up DRP objects like Clusters. This tutorial will explain how to create such a template, utilizing the Digital Rebar template actions provided by the backend plugin.

In this tutorial, we will be creating a Backstage template that walks a user through the process of spinning up a cluster in DRP. It is assumed that the steps in backend are followed so that the required custom actions are available.

Adding templates¶

This tutorial operates under the assumption that you have an understanding of how to create new templates and add them to your Backstage instance. This is described in the official Backstage documentation. A description of the template schema can also be found here.

Let's start by creating a directory to store our templates. From our Backstage instance...

mkdir -p packages/backend/templates/drp

Now, let's create our template.

touch packages/backend/templates/drp/create-cluster.yaml

Next, we need to register the template with our Backstage instance. This is described in the documentation link above.

Open app-config.yaml in the editor of your choice, and look for the catalog: object at the root of the configuration file.

By default, an example template is registered in a standard, unmodified Backstage installation. It looks like this:

catalog:
  # ...
  locations:
    # ...

    # Local example template
    - type: file
      target: ../../examples/template/template.yaml
      rules:
        - allow: [Template]

Let's add another one of those - type: file entries. It will reference the template we created.

- type: file
  target: templates/drp/create-cluster.yaml
  rules:
    - allow: [Template]

The template should now be registered in your Backstage instance, except our template is blank. Let's fill it out with our intended functionality.

Custom actions¶

The Backstage scaffolder/templates interact with the Digital Rebar API through custom actions. In the examples linked above, some actions include fetch:template, fetch:plain, publish:github, and catalog:register. These are actions that will be taken as steps are reached. The backend plugin is responsible for implementing these custom actions, which allow us to use things like creating clusters in our case.

Please complete the backend tutorial if you have not already.

Writing the template¶

Below is a blob of yaml source that we will set our template to. It will be broken down and explained afterwards. Set the template at packages/backend/templates/drp/create-cluster.yaml to the following source:

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: drp-create-cluster
  title: Create DRP Cluster
  description: Create a Digital Rebar Cluster.
spec:
  owner: user:guest
  type: service

  parameters:
    - title: Cluster information
      required: ["name", "broker"]
      properties:
        name:
          title: Name
          type: string
          description: The name of the Cluster to add.
          ui:autofocus: true
        broker:
          title: Broker
          type: string
          description: The broker for the Cluster.
  steps:
    - id: create
      name: Create Cluster
      action: drp:clusters:create # our custom action!
      input:
        # the following fields are directly
        # passed to the created DRP object
        Name: ${{ parameters.name }}
        Params:
          broker/name: ${{ parameters.broker }}
  output:
    links:
      - title: Jump to Cluster in UX
        url: https://portal.rackn.io/#/e/${{ steps.create.output.endpoint }}/clusters/${{ steps.create.output.object.Uuid }}

Now, let's look at each section of our template individually.

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: drp-create-cluster
  title: Create DRP Cluster
  description: Create a Digital Rebar Cluster.
spec:
  owner: user:guest
  type: service

The section above is simple template metadata. More information on this is described in Backstage documentation.

parameters:
  - title: Cluster information
    required: ['name', 'broker']
    properties:
      name:
        title: Name
        type: string
        description: The name of the Cluster to add.
        ui:autofocus: true
      broker:
        title: Broker
        type: string
        description: The broker for the Cluster.

The section above declares the parameters of the template. In our case, we need parameters for the name of the Cluster, and for the broker the Cluster should be assigned.

Again, documentation will help with adding more fields as necessary. You will likely want to include more parameters to construct more interesting objects, such as cluster size.

steps:
  - id: create
    name: Create Cluster
    action: drp:clusters:create # our custom action!
    input:
      # the following fields are directly
      # passed to the created DRP object
      Name: ${{ parameters.name }}
      Params:
        broker/name: ${{ parameters.broker }}

The steps object is responsible for declaring what the template actually does when it is constructed. Here, it calls the drp:clusters:create action we set up in the backend, and creates a preliminary DRP Cluster object with the data from our parameters.

  output:
    links:
      - title: Jump to Cluster in UX
        url: https://portal.rackn.io/#/e/${{ steps.create.output.endpoint }}/clusters/${{ steps.create.output.object.Uuid }}

Finally, the output object describes what is displayed to the user after the template is finished being constructed. Here, we have it set to show a link to the user, "Jump to Cluster in UX," that should link to the new object in the portal.

The ${{ steps.create.output.endpoint }} notation is templating syntax for the output of the create step (steps.create.output). Our endpoint field on this steps.create.output object is filled in by our custom action in the backend plugin, as is the steps.create.output.object object, which represents our new DRP Cluster as returned by the DRP API.

Usage¶

At this point, we have created and registered our template with our Backstage instance.

Before testing, ensure that you have set up authorization with DRP.

Spin up your Backstage instance...

$ yarn dev

Then click the Create button from the opened Catalog page. Your new template should be visible. Fill it out and create the Cluster!

Backstage Integration

Backend¶

Creating the plugin¶

Using the Digital Rebar TypeScript API¶

Adding custom actions¶

Adding router extensions¶

Exporting our plugin functionality¶

DRP Authorization¶

Register the plugin¶

If you made router extensions¶

If you added custom actions¶

Usage¶

Frontend¶

Creating the plugin¶

Adding the plugin page¶

ClusterActions¶

ClustersTable¶

Registering the plugin with Backstage¶

Register it in another instance¶

Usage¶

Scaffolder¶

Adding templates¶

Custom actions¶

Writing the template¶

Usage¶

`ClusterActions`¶

`ClustersTable`¶