How-to Set up a Batch Operations System

The purpose of this article to discuss how to build out a Batch Operations System.

A Batch Operations System is a framework that runs, manages, and reports on a set of batched actions.

Create a Batch Operations System's Environment

The easiest way to build a batch operations system's environment is to use the universal-application-batch-environment-cluster pipeline.

In the UX, you can create a cluster, name it, and select the universal-application-batch-environment-cluster pipeline. A name could be like event-worker-pool.

This will populate a set of fields to fill in.

• resource/broker - Use the context-broker
• docker-context/tag - the tag on the context worker nodes to use

Optionally, a context could be chosen to provide a specific toolset.

Once complete, the BOE is ready to handle Batch requests.

Batch Operation

An example Batch Object to create a batch looks like this:

---
SetupWorkOrderTemplate:
  Blueprint: batch-run
  Filter: "cluster/tags=In(event-worker-pool)"
  Params:
    batch/iterator: "Uuid"
    batch/iterator-param: "scan/machine-uuid"
    batch/filter-type: machines
    batch/filter: "Params.scannable=Eq(true)"
    batch/limit: 10000
    batch/order: sort=scan/last-scan-time
WorkOrderTemplate:
  Blueprint: audit-scan-me-simple
  Filter: "cluster/tags=In(event-worker-pool)"
PostWorkOrderTemplate:
  Blueprint: audit-complete-simple
  Filter: "cluster/tags=In(event-worker-pool)"

This batch object requires the batch content pack. It is uses a simple audit-scan-me-simple blueprint that doesn't do anything except sleep and mark the system scanned. It uses the audit-complete-simple blueprint to complete the batch.

It requires the machines to be marked as scannable by setting the scannable parameter to true.

It also requires the event-worker-pool cluster of context workers.

If more workers are needed, the cluster can be scaled up and the batch system will start using it.