Skip to content

Endpoint Architecture

Digital Rebar Provision is provided by a single binary that contains tools and images needed to operate. These are expanded on startup and made available by the file server services.

Services

Provisioning requires handoffs between multiple services as described in the workflows section. Since several of services are standard protocols (DHCP, TFTP, HTTP, HTTPS), it may be difficult to change ports without breaking workflow.

The figure below illustrates the four core Digital Rebar Provision services including protocols and default ports. The services are:

  1. API - These services provide control for the other services
    1. Service API: REST endpoints with Swagger definition
    2. UI: User interface redirect and local UX
    3. Websocket interface
    4. Swagger UI interface
  2. DHCP: Address management includes numerous additional option fields used to tell systems how to interact with other data center services such as provisioning, DNS, NTP and routing.
  3. Provision: sends files on request during provisioning process based on a template system:
    1. TFTP: very simple (but slow) protocol that is used by firmware boot processes because it is very low overhead.
    2. HTTP: faster file transfer protocol used by more advanced boot processes
    3. HTTPS: faster file transfer protocol used by more advanced boot processes
  4. RAFT: Synchronization and High Availability Support
    1. RAFT: Consensus protocol for maintaining state with the high availability cluster.

core_services.png

Ports

The table describes the ports that need to be available to run Digital Rebar Provision (DRP). Firewall rules may need to be altered to enable these services. The feature column indicates when the port is required. For example, the DHCP server can be turned off and that port is no longer required.

Ports Feature Usage
53/udp DNS DNS Port
53/tcp DNS DNS Port
67/udp DHCP DHCP Port
69/udp PROV TFTP Port
4011/udp BINL PXE/BINL port
8080/tcp Metrics Prometheus Metrics
8090/tcp PROV HTTPS-base Dynamic File Server
8091/tcp PROV HTTP-base Static File Server
8092/tcp Always HTTPS API and Swagger-UI
8093/tcp RAFT High Availability Protocol

All default ports can be changed at start up time of the dr-provision service. NOTE that changing DNS, DHCP and TFTP ports has wide ranging implications and is likely not a good idea (many firmware implementations can not be changed to use alternate port numbers).

Port access requirements:

In all provisioning usage cases (53, 67, 69, 4011, 8090, 8091, and 8092) the ports from the Machines being provisioned to the DRP Endpoint must be accessible. The DRP Endpoint must be able to reach the Machines being provisioned on port 67. In addition, the API and Swagger-UI port must be accessible to any operator/administrator workstations or systems that are controlling and managing the DRP Endpoint service. Additionally, any services or integrations that interact with the DRP Endpoint (eg IPAM, DCIM, Asset Management, CMS, CMDB, etc) may need access to the API port. BINL is an optional protocol and only needed if you are using it in place of PXE.

If you are running drpcli info status or drpcli info check from another host, make sure firewall rules are in place to allow communication between both the host and the endpoint as both commands check all ports on the DRP endpoint.

Optionally, the DRP Endpoint MAY use the HTTPS (443/tcp) or SSH port (22/tcp) to connect to machine remotely. This is an OUTBOUND port usage. It is a common usage for Terraform and Ansible execution from the DRP Endpoint.

Within an HA cluster, the DRP Endpoints MUST be able to bi-directionally communicate over the RAFT port. HA is an optional configuration.

Additionally, the DRP Endpoint can export Prometheus metrics, and by default metric service will run on port 8080. If you wish to scrape DRP metrics, you will need to accommodate this port as well.

Endpoint Firewall Configuration Examples

iptables

Here is an example of Linux based IPTables firewall rules. Note that you may need or be required to restrict the source IP addresses appropriately for your operational security requirements. For example, the Machines that will be provisioned need access to all ports, and administrators/operators may need access to the API port for control plane of the DRP Endpoint(s). You will need to adjust the input interface (-i eno3) appropriately for your system.

 # a vague attempt to make this roughly reusable...
 INTERFACE=eth0
 PORT_DHCP=67:68
 PORT_DNS=53
 PORT_TFTP=69
 PORT_SFILE=8090
 PORT_FILE=8091
 PORT_BINL=4011
 PORT_PROV=8092
 PORT_PROM=8080
 PORT_RAFT=8093
 SOURCES=""       # add appropriate "-s IP/netmask" etc statements here

 # for local connectivity
 iptables -A INPUT -i lo -p tcp -m tcp --dport $PORT_FILE -j ACCEPT
 iptables -A INPUT -i lo -p tcp -m tcp --dport $PORT_SFILE -j ACCEPT
 iptables -A INPUT -i lo -p tcp -m tcp --dport $PORT_PROV -j ACCEPT

 # adjust input interface appropriately
 iptables -A INPUT -i $INTERFACE $SOURCES -p udp --dport $PORT_DNS -j ACCEPT
 iptables -A INPUT -i $INTERFACE $SOURCES -p tcp -m tcp --dport $PORT_DNS -j ACCEPT
 iptables -A INPUT -i $INTERFACE $SOURCES -p tcp -m tcp --dport $PORT_FILE -j ACCEPT
 iptables -A INPUT -i $INTERFACE $SOURCES -p tcp -m tcp --dport $PORT_SFILE -j ACCEPT
 iptables -A INPUT -i $INTERFACE $SOURCES -p tcp -m tcp --dport $PORT_PROV -j ACCEPT
 iptables -A INPUT -i $INTERFACE $SOURCES -p udp --dport $PORT_BINL -j ACCEPT
 iptables -A INPUT -i $INTERFACE $SOURCES -p udp -m udp --sport $PORT_DHCP --dport $PORT_DHCP -j ACCEPT
 iptables -A INPUT -i $INTERFACE $SOURCES -p udp --dport $PORT_TFTP -m state --state NEW,ESTABLISHED -j ACCEPT
 iptables -A INPUT -i $INTERFACE $SOURCES -p tcp -m tcp --sport $PORT_RAFT -j ACCEPT
 iptables -A INPUT -i $INTERFACE $SOURCES -p tcp -m tcp --dport $PORT_RAFT -j ACCEPT

If your DRP Endpoint is listening on multiple network interfaces, you will need to adjust these rules appropriately.

firewalld

As root run the following command to enable all possible services used by DRP. Similar to the iptables instructions, it's important to consider security implications when enabling any of these ports.

firewall-cmd --permanent --add-port=8090-8093/tcp --add-port=67-69/udp --add-port=4011/udp --add-port=8080/tcp --add-port=53/tcp --add-port=53/udp

Guacamole Tunnel

Release v4.10 introduced the ability to create a console proxy via Guacamole. This allows operators to connect to the Digital Rebar Endpoint host and provisioned servers even if they do not have a direct network connection to them by creating a proxy through the API and then Guacamole service.

While Guacamole uses 4822 internally is does NOT require that port to be open. Instead, the Digital Rebar Endpoint tunnels console routes requests to the endpoint local Guacamole service via the API.

The connection path is as follows:

  1. User\'s Browser -->
  2. RackN Portal UX -->
  3. API call on 8092/upgraded to websocket -->
  4. DRP Endpoint -->
  5. localhost:4822/tcp connection -->
  6. guacd service container -->
  7. ssh to server in question

The ssh process requires that Digital Rebar has access to a private key that has been registered in the target server\'s authorized_keys list. Several Digital Rebar pipelines including the bootstrap and cloud provisioning processes automatically create key pairs.

Guacamole supports multiple console protocols: only ssh was supported in v4.10. We anticipate adding additional protocols in future releases.