Callback

The callback plugin allows for the definition of callbacks triggered by stages.

The callback adds an action to machines, callbackDo. This command takes one parameter, callback/action.

When the callbackDo action is called, the callbacks for the plugin are indexed by the the parameter, callback/action, and, if found, the plugin does the requested callback to the external service. If the action is not found, the action return an error.

The callback plugin will build a web request that will be sent to the external service. The callback can be defined by the structure. The callback structure is stored on the plugin in a map with the action name as the key for this structure.

The first set of fields define what API endpoint should be contacted and its retry, timeout, and delay configuration.

Name	Type	Description
Url	string	This is the full path to the service. This field is rendered. See below.
Method	optional string (POST)	method to use for the web request. e.g. POST, PUT
Headers	optional key/value pairs	These are passed directly as Request headers.
Retry	int (0)	How many retries to do before failing.
Timeout	int (7200)	Maximum time that this callback should take for all retries in seconds.
Delay	int (0)	How many seconds that should be delayed between retries.
NoBody	bool (false)	Should the body not be sent on the request
JsonResponse	bool (false)	Should the response be converted to a JSON blob. When true, this is always used.
StringResponse	bool (false)	Should the response be a JSON string of the raw response body
Debug	bool (false)	Enables logging of callback API request/response. Should be set to true along with the rs-debug-enable parameter.

The second set of fields define how authentication should be handled for this callback. The authentication methods can be created by name and these can be referenced for this callback. The Auth and Auths fields define a list of authentication methods to try to get a Bearer Authentication token. Auth was a single method and is retained for backwards compatibility. Auths allows for multiple methods to be tried in case others fail. The system will concatenate the two fields into a single list with Auth first. Both fields are options. Auth may be left empty and only Auths used.

Name	Type	Description
Auth	string	the first auth method to use
Auths	list of strings	additional auth methods to try.

The third set of fields define the content of the message sent for non-Get messages. By default, the system will send the machine object. The Aggregate and Decode fields will define if the Params field should be a full aggregate set and if the secure parameters should be sent decoded or not. Additionally, ExcludeParams can be used to remove parameters from the machine object to reduce the size of the message.

Name	Type	Description
Aggregate	boolean (false)	Include profile, stage, and global parameters for the machine object.
Decode	boolean (false)	Should secure parameters be decoded.
ExcludeParams	list of strings ([])	a list of parameters to exclude from the machine object.

It is also possible to replace the complete message with an override. Using the parameter, callback/data-override, an object can be sent instead. This is an untype and undefined parameter. This allows it to be defined as anything. If it is a non-string object, it will be JSON encoded and sent as the data field. If the value is a string, the string will be run through a template rendering engine much like the bootenv renderer. This allows for standard template injection. See the rendering section below.

If the web request returns an error code larger than or equal to 400, the action returns that error.

Otherwise, the action returns the body of the response.

The expected usage is to build a stage with the callback/action parameter. The callback-task is provided to use as the task for the stage. This stage is placed in a workflow at stages to indicate to external services what state a machine is in. Additional parameters can be added to the stage to augment the machine object that is posted to the external service.

Additionally, the callback plugin can be used to notify a callback when the machine's current job fails. If the callback map contains "jobfail" as a registered callback, the callback plugin will issue the callback with the machine object.

The callback tasks can be skipped by setting the callback/skip parameter to true. This applies to the default esxi and standard callback tasks. Custom tasks need to make sure to check that parameter for it to be applied.

Rendering functions that are available to the URL and data templating are similar to the bootenv rendering system. Here is a list of the functions:

Name	Description
Machine Fields	the fields on the Machine object are available to reference.
Param <parameter name>	returns the value of the Param defined by
ParamExpand <parameter name>	Returns the value of the Param after running through the renderer.
ParamAsYAML <parameter name>	Returns the parameter rendered as YAML.
ParamAsJSON <parameter name>	Returns the parameter rendered as JSON.
ParamExists <parameter name>	return true if the Param exists on the machine.
HexAddress	Returns the Hex form of the Machine.Address field
ShortName	Returns the first section of the Machine.Name (dot separated)
Path	Returns the machine's specific file path in the bootenv rendering space.
Url	Returns the machine's specific file path as a Fileserver URL.
HasProfile <profile name>	Returns true if the machine has the specified profile.
MacAddr <optional format>	Returns the first mac address on the machine in the format specified. Default is raw (conanical form). Other forms are pxe - 01-- separated> and rpi4 is mac separated by -.
ProvisionerAddress	The IP address of the DRP Endpoint.
ProvisionURL	the base URL for the fileserver space on the DRP Endpoint.
ApiURL	the base URL for the api server space on the DRP Endpoint.
Info	Returns the models.Info structure for the DRP Endpoint
ParseURL <segment> <url>	Returns the segment part of the specified URL
GetAuthString <name of auth method>	Returns the string or auth token from this auth method. This currently on supports the exec auth method.

All of these functions are referenced as . functions, e.g. .ApiURL.

While the plugin defines callbacks above, authentication methods can also be specified in the Auths map. The Auths map defines a set of named Auth structures. These auth structures look like the following:

Name	Type	Description
AuthType	string	One of basic, json-token, or exec.
Debug	bool (false)	Enables logging of Auth request/response. Should be set to true along with the rs-debug-enable parameter.

For exec types of Auth, the following fields are used.

Name	Type	Description
Path	string	the path to an executable to be executed to retrieve a single string. This string will have leading and trailing space removed. When used as a callback auth, the string will be assumed to be Bearer Auth token. When used for GetAuthString, the string will be inject in place as is.

For basic types of Auth, the following fields are used.

Name	Type	Description
Username	string	The username to use
Password	string	The password to use

For json-token type of Auth, the following fields are used.

Name	Type	Description
Url	string	URL of API Endpoint to request a json structure.
Method	string	Method for AUTH request
Data	string	Data to send on auth request - not templatized.
Query	string	QueryString to send for auth request - not templatized.
TokenField	string	Name of top-level field of JSON blob to get the token from.
DurationField	string	Name of top-level field holding an integer value that defines the duration in seconds of the token. This is optional.
Retry	int (0)	Number of retries before declaring a failure.
Timeout	int (7200)	Timeout total time in seconds for the auth request
Delay	int (0)	The number of seconds to wait between retries.

Upon successful, return the token will be cached for the DurationField time if specified.

Overriding Body Sent to the RESTfull API Endpoint

By default the Callback plugin is designed to send a full Machine Object as part of the Body of the RESTfull request (including GET type requests). In many cases, the operator may not want the full Machine Object, a partial subset of the Machine Object, or no Machine Object sent at all. In these cases, the use of the callback/data-override action setting can be used to shape what is provided in the Body of the request.

An example of using the Data Override capability:

# overrides the Body to an empty JSON blob
drpcli machines runaction Name:mach-07 callbackDo callback/action postman-get-test callback/data-override "{}"

In some cases, you may only wish to submit the Machine Object fields, without any of the Params, as they often can have extremely large data structures (eg bios-current-configuration, gohai-inventory, etc.). To override the object in the Body with just the Params stripped, use of jq (or a similar JSON transformation tool), as follows would minimize the

Body provided:

# strip the Params from the Machine Object
data=$(drpcli machines show Name:mach-07 | jq 'del(.Params)' )

# set the data-override to this slimmed down object
drpcli machines runaction Name:mach-07 callbackDo callback/action postman-get-test \
  callback/data-override "$(cat data | jq -sR .)"

Similarly - you can set the value of our $data variable above to any combination of JSON data needed for your use case. This includes manipulation of the Machine Object, or completely replacing it with other data.

Example Usage: Postman-Echo

The callback plugin is designed to provide generic access to RESTful services. Here is an example usage of the Callback plugin.

Note

This example uses the publicly accessible Postman-Echo service. This service is not very stable, and it appears to be globally rate-limited. This means that if you are testing the configuration and usage, you may have to run the callbackDo action several times before you get a valid response.

This example usage scenario will utilize a simple RESTful GET request to the Postman-Echo service, at the URL:

• https://postman-echo.com/

We will pass in two fictional Query String Params ("foo1" and "foo2"). The Digital Rebar Platform (DRP) Callback plugin has a few default behaviors that we will have to override to shape the return response the way we might expect to see it.

Here is a sample of what the equivalent curl call would be that this example is based on:

curl --silent --request GET "https://postman-echo.com/get?foo1=bar1&foo2=bar2" | jq '.'

Making the Callback call, the following changes to the default behaviors will be made in this

Example:

• we will override the default Body, which typically is a full Machine Object
• the Callback plugin returns a base64 encoded blob (for historical reasons), we will need to decode this

Install the Plugin Provider

You must first install the Callback Plugin Provider via the Web Portal, or via the command line. To install via the Web Portal, visit the Catalog menu time, type "callback" in the search field, then select the most recent stable version and click install.

For installation via the command line, simply do:

# installs most recent Stable version of Callback
drpcli catalog item install callback

Once you have the Plugin Provider installed, you will then instantiate a Plugin with the appropriate configuration (by setting Params on the Plugin) for the RESTful service you wish to interact with. See the following sections for these details.

Create the Plugin Configuration

The Plugin instantiated from the Plugin Provider is configured to interact with the various RESTful services by setting values to the various Params used to configure it. The Plugin is added in the Web Portal via the Plugins menu options. This example will focus on use of the CLI (drpcli) to create the configurations along with instantiating the Plugin.

Save the following JSON structure as a local file, in this example we will use the filename callback-postman-echo-get.json.

{
  "Name": "callback-postman-echo",
  "Description": "Tests RESTful APIs against Postman-Echo service.",
  "Provider": "callback",
  "Meta": {
    "color": "orange",
    "icon": "bullhorn",
    "title": "Postman-Echo"
  },
  "Params": {
    "callback/callbacks": {
      "postman-get-test": {
        "Method": "GET",
        "Retries": 3,
        "Headers": {
          "Accept": "application/json",
          "Content-Type": "application/json"
        },
        "Timeout": 60,
        "Url": "https://postman-echo.com/get?foo1=bar1&foo2=bar2"
      }
    }
  }
}

Once you have your Plugin configuration JSON file ready, you will create the Plugin with the following syntax:

drpcli plugins create callback-postman-echo-get.json

Please observe the return JSON data structure to verify the results. Specifically, check the Errors field for any validation/configuration failures.

If you are updating your configuration and reloading new settings, you will need to either destroy the Plugin first, and then do a new create, or you can use the update method, as follows (Note you must specify the defined plugin Name for the update):

# example update usage
drpcli plugins update callback-postman-echo callback-postman-echo-get.json

# example destroy/create usage
drpcli plugins destroy callback-postman-echo
drpcli plugins create callback-postman-echo-get.json

Using the Postman-Echo Example

Plugin actions are similar to Machine actions; they must be executed in the context of a Machine that has a valid Runner (Agent) running. Below is an example of how to use the Callback action. This is an example which an operator might wrap in a Stage/Task/Template to be used as part of Workflow constructs to interact with RESTfull infrastructure services, or with any RESTfull API endpoint.

You can test the operation of the Plugin actions via the CLI, but remember you must have a Machine that is running the Agent (Runner). Example usage of our above "postman-echo-test" action:

###
#  WARNING - this is probably not quite right, see below descriptions and information.
#            Change the Name: to reflect machine name, or the Machines UUID (if in
#            content template, use '{{ .Machine.UUID }}')
###

drpcli machines runaction Name:mach-07 callbackDo callback/action postman-get-test

Below are some explanations and reasons why the above command is probably not what you want or need.

Note

As of Plugin Provider* v4.3.0 version of the Callback plugin, there are new options available to change these behaviors as part of the configuration of the Plugin itself. Please consult the documentation for these values.

By default the DRP Callback plugin is designed to return a Base64 encoded blob (for historical reasons). If you need to view/interpret the results of the return data, you will need to do a Base64 decode operation. See below for an example. This can also be altered by callback configuration options. JsonResponse set to true will return a json blob. StringResponse set to true will return a JSON string of the raw response body.

Additionally, the Callback plugin will also provide a full Machine Object as part of the Body of the request; even for GET type requests. If you wish to strip or modify the Body, you can do so at the time of the callbackDo action with the use of the callback/data-override setting. This can be altered by the NoBody flag on the callback configuration. This will ignore all body options and send no body.

In our example use case, it's not relevant to post the Machine Object to the Postman-Echo service, so we should strip it out of the request. The below example will use a command line pipline to perform the following transforms on our request:

• Set the Body to an empty JSON string ("{}") - stripping the Machine Ojbect from being sent
• Pass the return result from Postman-Echo to base64 decode it (note the object is wrapped in double quotes, which we have to strip first)
• Finally, JSON pretty-print the output via the jq command

# below should be one single command line - change the Name: to reflect machine
# name, or the Machines UUID (if in content template, use '{{ .Machine.UUID }}')

drpcli machines runaction Name:mach-07 callbackDo callback/action postman-get-test \
  callback/data-override "{}" | sed 's/"//g' | base64 -d - | jq '.'

Note

Do not attempt to set the callback/data-override value in the Plugin configuraiton, it is only valid as part of the action call.

The returned results should look similar to the following:

{
  "args": {
    "foo1": "bar1",
    "foo2": "bar2"
  },
  "headers": {
    "x-forwarded-proto": "https",
    "x-forwarded-port": "443",
    "host": "postman-echo.com",
    "x-amzn-trace-id": "Root=1-5eb99202-6b5f0f7779cfb8d84b476aba",
    "content-length": "2",
    "user-agent": "Go-http-client/1.1",
    "accept": "application/json",
    "content-type": "application/json",
    "accept-encoding": "gzip"
  },
  "url": "https://postman-echo.com/get?foo1=bar1&foo2=bar2"
}

Warning

WARNING - the Postman-Echo service regularly rate-limits requests (globally). If you receive an output of Error:, wait a moment, and try again. Sometimes you may need to try several times before getting a response.

The above callbackDo action now matches our returned JSON data structure we would get back from our example

curl call listed above.