Digital Rebar Platform: Remote API Integration — Architect Guide¶

Architecture & Design Reference

1. Overview¶

Automating interactions with remote API services is a foundational capability of the Digital Rebar Platform (DRP) Pipeline provisioning system. Before writing a single line of automation logic, several architectural decisions must be made — decisions that determine how your workflow handles timing, data flow, errors, and operational reliability.

This document is intended for DRP Architects designing Pipeline automation systems. It covers the concepts, decision frameworks, and design principles that govern remote API integration. For implementation patterns and working code examples, see the companion API Integration Developer Guide .

2. Where DRP Automation Tasks Run¶

Understanding where a task executes is critical when automating interactions with remote API services, because the execution context determines network reachability, trust level, and available credentials.

DRP tasks run in one of three contexts:

Machine Runner (Agent)

The task executes directly on the Machine being provisioned, under the operating system present at automation time — typically either Sledgehammer (the DRP in-memory discovery and staging environment) or the Machine's installed OS. Scripts run as a process on the Machine itself.

Plugin Action

The task executes as a Plugin action from within the DRP Endpoint service process itself. The call originates from the DRP Endpoint host rather than from the Machine being provisioned.

Context Container

The task executes inside an ephemeral container machine that DRP creates and destroys specifically for the duration of the context switch. The Machine's runner temporarily hands off execution to the context container, which runs on the DRP Endpoint host, and control returns to the Machine runner when the context task completes.

3. Foundational Architectural Decisions¶

Every API integration in a DRP Pipeline begins with five core questions. Answering them before writing code prevents entire categories of bugs and operational incidents.

1. What Inputs Does the Service Require?
2. Synchronous vs. Asynchronous Execution
3. Response Parsing and State Capture
4. Idempotency
5. Error Handling

3.1 What Inputs Does the Service Require?¶

Before any other design decision can be made, you must fully understand what the remote service needs to accept and process your request. This is the prerequisite to everything else — you cannot reason about payload format, authentication, or error handling until you know what you are sending.

API Protocol and Interaction Mechanism¶

DRP supports a wide range of API protocols. While RESTful HTTP services are the easiest to work with and the most common, your automation may need to interact with other service types:

Payload Format¶

For RESTful services, DRP strongly prefers JSON payloads (Content-Type: application/json). JSON is native to DRP's internal data model, making it straightforward to construct payloads from Machine or Profile Parameters using jq, and equally straightforward to parse responses back into Parameters. Other formats are supported when the target service requires them, but they introduce additional parsing complexity.

When sending data to a RESTful service, the HTTP method and payload strategy depend on the operation being performed:

JSON Patch and the test operation

When using PATCH, prefix your patch document with a test operation whenever the safety of the update depends on the current state of the resource. The test operation causes the server to reject the entire patch atomically if the specified field does not match the expected value, preventing race conditions and unintended overwrites.

Required Inputs Checklist¶

Before designing your task script, gather and document the following for the target API:

• Endpoint URL — base URL, path structure, and any path Parameters (e.g., resource IDs)
• Authentication mechanism — bearer token, API key, client credentials, basic auth
• Required headers — Content-Type, Accept, custom service headers
• Payload schema — required and optional fields, data types, enumerated values
• Source of each input value — which DRP Machine Parameter, Profile Parameter, or Global Parameter supplies each field
• Encoding and format constraints — date formats, string length limits, allowed characters

3.2 Synchronous vs. Asynchronous Execution¶

Once you know what you are sending, the next question is whether the action completes before the API responds, or whether the API merely acknowledges receipt of the request and performs work separately.

Synchronous (fire-and-confirm)

The remote service performs the work inline and returns a final result in a single HTTP response. The Pipeline stage can read the result immediately and continue or fail.

Asynchronous (fire-and-poll)

The remote service accepts the request, returns a job ID or a reference URL, and then performs the work in the background. Your Pipeline stage must block and poll the service repeatedly until it reports completion or failure.

3.3 Response Parsing and State Capture¶

Receiving an HTTP 200 status code does not mean the operation succeeded. Many services return 200 with an error payload. Conversely, some return 201 or 202 for genuinely successful asynchronous submissions. Your automation must inspect the response body to determine actual outcome.

Beyond success/failure determination, responses often contain data that is valuable for downstream Pipeline stages. Examples include:

• Assigned IP addresses or hostnames returned after provisioning a network resource
• Job or ticket IDs needed to poll asynchronous status
• Bearer tokens or session identifiers required for subsequent API calls
• Version strings or configuration identifiers that should be recorded for audit purposes

DRP stores this kind of captured state as Parameters on the Machine or Profile object, making it available to every subsequent Stage in the Pipeline. Use drpcli or the DRP REST API to write these values back as soon as they are captured.

3.4 Idempotency¶

A DRP Pipeline task may execute more than once against the same Machine. Network interruptions, agent restarts, operator retries, and Pipeline re-runs can all cause a task to fire again after it has already succeeded — or partially succeeded. The interaction with the remote API service must therefore be designed to be idempotent: running the same operation multiple times must produce the same end state as running it once, with no duplicate side effects.

Idempotency is not a property the remote API automatically provides — it must be deliberately designed into your task logic.

Idempotency for synchronous operations follows a check-before-act pattern: query current state, compare against desired state, and only issue the change if needed. Treat "already in desired state" as a success.

Idempotency for asynchronous operations requires storing the job ID as a DRP Parameter immediately after submission. On every subsequent run, check for an existing job ID before submitting a new one — if one exists, resume monitoring it rather than creating a duplicate. On job completion, clear the job ID Parameter so an intentional future re-run can start fresh.

Idempotency Considerations by HTTP Method¶

3.5 Error Handling¶

Remote services fail. Networks partition. Rate limits are hit. Authentication tokens expire. Robust automation treats these conditions as normal operating circumstances, not edge cases.

At minimum, every API call in a DRP Pipeline should:

• Capture the HTTP status code and distinguish between client errors (4xx) and server errors (5xx).
• Parse any error detail from the response body and surface it in DRP task output for operator visibility.
• Distinguish between terminal failures (e.g., 403 Forbidden — retrying will not help) and transient failures (e.g., 503 Service Unavailable — retrying is appropriate).
• Set the DRP task result to a failed state explicitly when an unrecoverable error is detected.

3.6 Retry Logic¶

For transient failures, a retry loop with exponential backoff is the standard pattern. The key Parameters are:

4. DRP Platform Preferences for API Interactions¶

4.1 Preferred Service Type: RESTful JSON APIs¶

DRP automation is optimized for RESTful HTTP services that accept and return JSON payloads. JSON is the easiest format to parse in bash using tools such as jq, which is available in the DRP runner environment. This combination — REST + JSON + jq — allows clean extraction of individual fields without fragile text parsing.

Other service types are supported when necessary. SOAP/XML services, proprietary binary protocols, and form-encoded endpoints can all be called from DRP task scripts, but they require additional parsing effort and should be wrapped in helper functions to keep Pipeline task scripts readable.

4.2 Preferred Update Method: JSON Patch¶

When updating an existing remote resource, prefer JSON Patch operations (RFC 6902) over sending a complete replacement payload. JSON Patch is more explicit, less error-prone, and reduces the risk of inadvertently overwriting fields that were not intended to change. It also makes idempotency easier to reason about, since each operation targets a specific field at a specific path.

5. Decision Guide Summary¶

Use this table as a quick reference when designing a new API integration in a DRP Pipeline Stage.

6. DRP-Friendly Service Design Checklist¶

The following checklist describes behaviors that make a remote API service easy to integrate with DRP Pipeline automation. It is intended as a design guide for teams building or evaluating services that DRP will orchestrate — not as a requirement for services outside your control.

Interface & Protocol¶

• Exposes a RESTful HTTP/HTTPS interface
• Accepts and returns JSON (application/json) as the primary content type
• Supports JSON Patch (application/json-patch+json) for partial resource updates
• Supports the JSON Patch test operation for conditional, atomic updates
• Uses standard HTTP methods semantically (GET, POST, PUT, PATCH, DELETE)
• Returns meaningful and consistent HTTP status codes (not 200 for all responses including errors)

Inputs & Discoverability¶

• Documents all required and optional fields for every endpoint
• Accepts resource identifiers (IDs, names) that are stable and externally assignable, so DRP can reference resources predictably across runs
• Does not require session state or multi-step handshakes before accepting operational requests

Responses & State Visibility¶

• Returns structured error responses in JSON with a machine-readable error code and human-readable message
• Distinguishes between client errors (4xx) and server errors (5xx) correctly
• Returns the current state of a resource on GET in a consistent, stable schema
• Includes a resource's current status as a queryable field so automation can verify desired state before acting

Asynchronous Operations¶

• Returns a stable job or operation ID for any long-running action
• Exposes a dedicated status endpoint queryable by job ID
• Reports a clear terminal status value for completed, failed, and cancelled jobs (not just the absence of a running state)
• Includes error detail in the terminal failure response, not only a status code

Idempotency¶

• PUT and PATCH operations are idempotent by design
• POST operations either are idempotent or support a client-supplied idempotency key header (e.g., Idempotency-Key)
• Repeated submission of the same request does not create duplicate resources or trigger duplicate side effects
• Accepts a DELETE against an already-deleted resource with 200 or 404 rather than an error

Authentication & Security¶

• Supports a standard authentication mechanism (OAuth 2.0 bearer token, API key, or mTLS)
• Returns 401 Unauthorized for missing or invalid credentials and 403 Forbidden for insufficient permissions — not a generic 400 or 500
• Supports token-based auth with configurable expiry so DRP can acquire short-lived credentials at Stage start

Rate Limiting & Reliability¶

• Returns 429 Too Many Requests when rate limits are exceeded
• Includes a Retry-After header on 429 responses indicating the correct backoff duration
• Returns 503 Service Unavailable (not 500) for transient overload conditions, signalling that a retry is appropriate

Digital Rebar Platform | Remote API Integration — Architect Guide | For internal and customer use