TFTP Debugging FAQ¶

Answers to the questions that come up most often when a machine will not network boot and TFTP is suspected.

For a linear walk through the whole PXE boot sequence, start with PXEBoot and TFTP Troubleshooting instead. For the architecture, see How dr-provision Boots Machines.

Confirming Requests Reach DRP¶

How do I confirm a machine's TFTP request actually reached DRP?¶

Watch the TFTP event stream, then power on the machine:

drpcli events watch "tftp.serve.*"

DRP publishes one tftp.serve event per request, keyed on the requested filename:

{
  "Time": "2026-07-31T14:02:11.412554192Z",
  "Type": "tftp",
  "Action": "serve",
  "Key": "lpxelinux.0",
  "Principal": "tftp",
  "Object": {
    "Start": "2026-07-31T14:02:11.402554192Z",
    "End": "2026-07-31T14:02:11.412554192Z",
    "RequestSize": 0,
    "ResponseSize": 45056,
    "Status": "SUCCESS",
    "Requestor": "10.10.20.76",
    "Url": "lpxelinux.0"
  },
  "Original": null
}

Requestor identifies the client, Status is SUCCESS, FAILED, or CRASHED, and Start/End bracket the transfer so you can spot slow or stalling fetches. RequestSize is always 0 for TFTP reads. See File Transfer Events for every field.

No special role claim is required to watch tftp events. Any authenticated principal will receive them.

Narrowing the stream¶

Add a fourth component to the specifier to filter on any Object field:

# Everything one machine asks for
drpcli events watch 'tftp.serve.*.Requestor=Eq(10.10.20.76)'

# Aborted and failed transfers, benign size probes included
drpcli events watch 'tftp.serve.*.Status=Eq(FAILED)'

Filter this way rather than putting the filename in the key field. Specifiers are split on ., and TFTP keys are filenames that usually contain a dot, so tftp.serve.lpxelinux.0 does not match lpxelinux.0 (the trailing 0 is parsed as a filter expression). There is no working way to register an exact key containing a dot, so always register tftp.serve.* and narrow with a filter.

See Filtering Subsystem Events for the filter grammar and the constraints that apply to tftp events.

The event watch is silent. What does that mean?¶

No tftp.serve events at all means DRP received no TFTP request. Work through these in order:

1. Confirm TFTP is enabled and on the port you expect.

   Bash

   drpcli info get | jq '{tftp_enabled, tftp_port, prov_enabled, address}'
   

   tftp_enabled: false means the endpoint was started with --disable-tftp or RS_DISABLE_TFTP_SERVER.

2. Confirm the TFTP server actually responds.

   Bash

   drpcli info status
   

   This is not a port scan. info status performs a real TFTP fetch of lpxelinux.0 against the endpoint, so "TFTP": { "Enabled": true, "Alive": true, "Port": 69 } in the reported service list means the full round trip works from wherever you ran drpcli. Its Enabled combines tftp_enabled and prov_enabled, so a false here can mean the whole provisioner is off rather than just TFTP. Check both fields from step 1.

3. If info status reports Alive: true but booting machines produce no events, TFTP is healthy and the request is being lost between the machine and DRP. See What capture commands should I run? and capture on the DRP host.

4. Check that the machine was told to use TFTP at all. UEFI HTTP Boot clients fetch their bootloader over HTTP and never touch TFTP. You will see static.serve events instead. See Can I avoid TFTP entirely?.

DHCP has no event stream¶

A machine that never receives a next-server and boot filename will never send a TFTP request, and the TFTP event stream cannot tell you that. DHCP publishes no events, so raise its log level instead:

drpcli prefs set debugDhcp debug
drpcli logs watch
# and afterwards, back to the default
drpcli prefs set debugDhcp warn

If the DHCP logs show no request either, capture both protocols together as shown in What capture commands should I run?. See also DHCP Architecture and External DHCP Architecture.

How do I turn up TFTP logging?¶

The TFTP server logs through the static file service, so the preference is debugStatic.

drpcli prefs set debugStatic debug
drpcli logs watch

TFTP lines carry Service static and Principal tftp, so you can isolate them instead of reading the whole stream:

drpcli events watch 'log.*.static.event.Principal=Eq(tftp)'

The messages worth looking for:

Return the preference to warn afterwards, as debug on a busy endpoint is noisy:

drpcli prefs set debugStatic warn

See Log Events for the event payload and the full list of log services.

Packet Capture¶

Why does tcpdump port 69 show the request but no file data?¶

Because TFTP only uses port 69 for the first packet of each transfer.

RFC 1350 has the server answer from a freshly allocated port. For every read or write request it opens a new UDP socket on an ephemeral port, bound to the local address the request arrived on. All option acknowledgements, data blocks, ACKs, and error packets flow between the client's ephemeral port and that new server port.

Two consequences when capturing:

• port 69 captures only the request. You see the client ask and nothing else, which looks identical to a server that never replied.
• Data-port traffic is not decoded. tcpdump only applies its TFTP decoder when a port is 69, so everything on the ephemeral port prints as a bare UDP, length N even when you do capture it.

Filter on the client's address instead of the port:

tcpdump -i any -n -vv 'host 10.10.20.76 and udp'

What capture commands should I run?¶

Run these on the DRP endpoint unless noted. All require root.

Watch the DHCP handshake and the TFTP request together:

tcpdump -i any -n -vv 'udp port 67 or udp port 68 or udp port 69'

Capture one machine's entire boot, including every data block, to a file for later analysis:

tcpdump -i any -n -s0 -w /tmp/tftp-boot.pcap 'host 10.10.20.76'

-i any matters on a multi-homed endpoint, where you may not know which interface DRP will answer on, and -s0 is required for the data blocks to be reassemblable. Open the capture in Wireshark and decode the data-port conversation as TFTP. tcpdump has no "decode as" option, so on replay those packets still print as UDP, length N.

Confirm from the client side of the network whether TFTP works at all, without waiting for a machine to boot. drpcli has a built-in TFTP client, so there is nothing to install:

drpcli static download lpxelinux.0 to /tmp/lpxelinux.0 --tftp

Run that from a host on the same VLAN as the machines that are failing, with tcpdump running on the DRP endpoint. If the download succeeds, the network path is fine.

What does a healthy transfer look like?¶

A BIOS PXE client at 10.10.20.76 fetching the bootloader from DRP at 10.10.20.10:

14:02:11.402554 IP 10.10.20.76.2070 > 10.10.20.10.69: TFTP, length 28, RRQ "lpxelinux.0" octet tsize 0
14:02:11.403004 IP 10.10.20.10.45123 > 10.10.20.76.2070: UDP, length 14
14:02:11.403981 IP 10.10.20.76.2070 > 10.10.20.10.45123: UDP, length 17
14:02:11.404881 IP 10.10.20.76.2075 > 10.10.20.10.69: TFTP, length 33, RRQ "lpxelinux.0" octet blksize 1456
14:02:11.405233 IP 10.10.20.10.45124 > 10.10.20.76.2075: UDP, length 15
14:02:11.406102 IP 10.10.20.10.45124 > 10.10.20.76.2075: UDP, length 1460
14:02:11.406540 IP 10.10.20.76.2075 > 10.10.20.10.45124: UDP, length 4

Reading it line by line:

1. First RRQ with tsize 0: the client is asking how big the file is, not for the file.
2. DRP answers with an OACK carrying the size, from the new port 45123. Undecoded because the port is not 69.
3. The client sends an ERROR and drops the transfer. It has what it wanted.
4. Second RRQ, from a different client port, this time negotiating blksize. This is the real transfer.
5. OACK confirming the block size, from another new server port 45124.
6. First data block: 4 bytes of TFTP header plus 1456 bytes of payload.
7. The client ACKs block 1. Steps 6 and 7 then repeat to the end of the file.

Timestamps, ephemeral port numbers, and packet lengths will differ in your capture. The shape is what matters: two requests per file, replies from ports other than 69, and a steady alternation of data and ACK packets.

What do the common failures look like on the wire?¶

Pair every capture with drpcli events watch "tftp.serve.*". The capture tells you what was on the wire, and the event tells you what DRP processed it as.

The transfer starts, then stalls and retries the same block¶

This is almost always an MTU mismatch, and it is the most common cause of a PXE boot that gets partway through a kernel and then hangs.

TFTP over UDP has no path MTU discovery. When a client requests a blksize option, DRP grants up to the MTU of the interface the request arrived on, minus 32 bytes for IP, UDP, and TFTP headers, with a floor of the 512-byte protocol default. On a standard 1500-byte interface that is up to 1468 bytes per block.

If any hop between DRP and the machine has a smaller MTU (a tunnel, a VPN, a VXLAN overlay, or a jumbo-frame-enabled DRP interface talking to a 1500-byte switch path) those blocks are dropped. DRP retries each block 5 times at 5-second intervals, so a black-holed transfer takes roughly 25 to 30 seconds to give up rather than failing fast. The signature in a host-filtered capture is a data packet, then the same UDP length repeating every 5 seconds with no ACK between.

# What MTU is DRP's provisioning interface using?
ip link show <interface>

# Does a full-size frame reach the machine's network? (1472 + headers = 1500)
ping -M do -s 1472 -c 3 10.10.20.76

If the ping fails while a smaller size succeeds, you have found the constrained hop. Either raise the MTU along the path or lower it on DRP's provisioning interface so the granted block size fits. Because the grant is derived from the interface MTU at request time, lowering the DRP interface MTU is the fastest way to prove the diagnosis without touching the network.

The reply comes from an unexpected source address¶

DRP replies from whichever local address the request arrived on, and caches that client-to-local mapping so rendered templates carry a ProvisionerURL the machine can reach. Strict clients reject a reply whose source differs from the address they sent to, which shows up on multi-homed endpoints, on HA pairs behind a virtual IP, and wherever routing is asymmetric.

ss -ulnp | grep :69
drpcli subnets show <subnet-name> | jq '{NextServer, Options}'

Compare the destination the client used against the source DRP replied from in the capture. An empty NextServer means DRP nominates itself, which is correct for most single-homed deployments. See Preparing to Run DRP for the routing and multi-homing implications.

What Should Be Transferring¶

Which files come over TFTP and which come over HTTP?¶

This depends entirely on which bootloader the machine is using, and it is the single most useful thing to know before reading a capture. A UEFI Secure Boot machine pulls its whole kernel and initrd over TFTP, while an iPXE machine pulls almost nothing over it.

The bootloader binary is named by DHCP option 67. The defaults come from the bootloaders param (lpxelinux.0 for 386-pcbios, ipxe.efi for amd64-uefi, and ipxe-arm64.efi for arm64-uefi) and a BootEnv's Loaders field overrides them. The sledgehammer BootEnv, for example, sets a shimx64.efi loader for amd64-uefi so that UEFI Secure Boot works, which routes the kernel and initrd over TFTP as a side effect.

Configuration files are requested in a waterfall, most specific first, so seeing requests for files that do not exist is expected:

Kernel and initrd paths for a machine-scoped render look like machines/<machine-uuid>/boot/<sha>/vmlinuz0. To see exactly what DRP will serve a given machine, inspect its BootEnv:

drpcli machines show <uuid> | jq '{BootEnv, Arch, Address}'
drpcli bootenvs show sledgehammer | jq '{Loaders, OS}'

Can I avoid TFTP entirely?¶

Partly, and it is often the right answer when TFTP is blocked by policy or performing badly over a long or lossy path.

• UEFI HTTP Boot removes TFTP from the boot path completely. Such a machine sends DHCP vendor class HTTPClient (option 60) instead of PXEClient, and DRP rewrites the boot filename into an http:// URL. Requests then arrive as static.serve events. Enable it in the machine's UEFI firmware; DRP needs no configuration change.
• iPXE keeps TFTP for the bootloader and config script but moves the kernel and initrd to HTTP, which is where nearly all the bytes are. This is the default for amd64-uefi PXE clients.
• Legacy BIOS PXE and UEFI Secure Boot (shim/grub) cannot avoid TFTP. Either fix the path, or boot from virtual media with the boot-virtual-iso param, which requires the IPMI plugin.

References¶

• PXEBoot and TFTP Troubleshooting: step-by-step walk through the whole PXE boot sequence
• Discovery Troubleshooting: other discovery troubleshooting topics
• How dr-provision Boots Machines: how DRP decides what to serve a booting machine
• PXE: PXE discovery architecture
• DHCP Architecture and External DHCP Architecture: DHCP architecture, internal and external
• Subsystem Events: tftp.serve and static.serve event reference
• Client compatible filter language: event filter expression grammar
• TFTP Error: the benign TFTP Aborted messages
• lpxelinux.0 error: no such file or directory: the benign pxelinux.cfg waterfall messages
• THE TFTP PROTOCOL (REVISION 2)
• TFTP Blocksize Option