Skip to content

Point builds at miren's bridge DNS instead of public DNS - #1103

Open
evanphx wants to merge 1 commit into
mainfrom
evan/mir_1695-builds-resolve-dns-against-host-resolver
Open

Point builds at miren's bridge DNS instead of public DNS#1103
evanphx wants to merge 1 commit into
mainfrom
evan/mir_1695-builds-resolve-dns-against-host-resolver

Conversation

@evanphx

@evanphx evanphx commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Problem

Container image builds silently resolve DNS against public servers (8.8.8.8 / 8.8.4.4) instead of the host's configured resolver. On a host that uses a systemd-resolved stub — nameserver 127.0.0.53 in /etc/resolv.conf — with an internal-only or egress-filtered upstream, builds can't resolve names at all. Reported by a user whose build environment couldn't do DNS.

Root cause

Build RUN steps run in the host network namespace (with no CNI config, buildkit's "auto" mode falls back to host networking), but buildkit generates each build's resolv.conf as if it were sandboxed. The step's NetMode is UNSET, not HOST, so GetResolvConf strips every loopback nameserver (the whole 127.0.0.0/8 range, 127.0.0.53 included) and, finding none left, substitutes 8.8.8.8/8.8.4.4. It can't recover the real upstream either, since /run/systemd/resolve/resolv.conf isn't mounted into buildkitd — only the host's /etc/resolv.conf is, via oci.WithHostResolvconf.

So even though the build sits in the very netns where 127.0.0.53 would resolve, buildkit discards it and hardcodes public DNS — the "breaks builds on hosts with an internal resolver" case the earlier MIR-1643 note meant to avoid.

Fix

Add a [dns] section to the generated buildkitd.toml pointing at miren's own bridge DNS server, wired in from the network boot node's router address. That resolver already serves runtime sandboxes: it forwards to the host's real upstream and answers *.app.miren. The bridge gateway is a local interface in the host netns, so build steps reach it, and buildkit keeps it (not a loopback, so the filter leaves it alone). External buildkit daemons are untouched.

Verification

  • Unit tests (TestGenerateConfig) and server boot/startup tests pass; golangci-lint run ./... reports 0 issues.
  • End to end in the dev env — rebuilt, restarted, ran a real Dockerfile build:
    • Before: RUN cat /etc/resolv.confnameserver 8.8.8.8
    • After: nameserver 10.8.13.1, and nslookup example.com resolves through it (exit=0)

Fixes MIR-1695

Build RUN steps run in the host network namespace, but buildkit
generates their resolv.conf as if they were sandboxed: it strips the
host's loopback stub resolver (systemd-resolved's 127.0.0.53) and
substitutes public DNS (8.8.8.8). On a host with an internal-only or
egress-filtered resolver, builds then can't resolve names at all.

Add a [dns] section to the generated buildkitd.toml pointing at miren's
own bridge DNS server, wired in from the network boot node's router
address. That resolver already serves runtime sandboxes: it forwards to
the host's real upstream and answers *.app.miren. The bridge gateway is
a local interface in the host netns, so build steps can reach it, and
buildkit keeps it since it isn't a loopback address.

Fixes MIR-1695
@evanphx
evanphx requested a review from a team as a code owner August 30, 2026 04:17
@coderabbitai

coderabbitai Bot commented Aug 30, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 037b4f25-8099-4ec5-b0c9-4f26e7ea12c1

📥 Commits

Reviewing files that changed from the base of the PR and between 22ff0e6 and 3989251.

📒 Files selected for processing (4)
  • components/buildkit/buildkit.go
  • components/buildkit/config_test.go
  • components/server/boot_buildkit.go
  • components/server/startup.go

Included review availability: 5 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 6 reviews per hour.


📝 Walkthrough

Walkthrough

BuildKit configuration now accepts optional DNS nameservers. When configured, generated TOML includes a quoted [dns] section. Embedded BuildKit startup derives the nameserver from a valid network router address. Invalid router addresses produce no DNS configuration. Startup wiring now passes network boot output to the BuildKit boot component.

Merge Risk: ⚪ Minimal · up to 39892

Builds will use Miren’s bridge DNS instead of public resolvers, restoring name resolution for hosts with internal DNS configurations. No actionable merge-blocking risk remains beyond normal checks and review.


Comment @coderabbitai help to get the list of available commands.

@miren-code-agent miren-code-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍪 biscuit: ✅ ready to merge — auto-review, non-blocking

I read through all four changed files and traced the full path from networkBootOutput.routerAddress through to the generated buildkitd.toml. This is a tight, focused fix with a clear problem statement.

The problem is real and the fix is correct. BuildKit strips loopback nameservers (the whole 127.0.0.0/8 range) and falls back to 8.8.8.8 when constructing resolv.conf for build steps. On hosts using systemd-resolved's 127.0.0.53 stub this silently breaks builds behind internal or egress-filtered resolvers. Pointing the [dns] section at miren's bridge router address — a non-loopback address reachable from the host netns — routes around this correctly.

Implementation details I checked:

  • network.routerAddress.IsValid() guards the zero-value case before constructing the nameserver list. If the network boot hasn't set a router address for any reason, dnsNameservers stays nil and generateConfig emits no [dns] section, leaving buildkit's default behavior intact. Safe fallback.
  • Nameserver quoting uses fmt.Sprintf("%q", ns), which for netip.Addr.String() output (a well-formed IP address) produces valid TOML string literals. No injection risk since the value comes from a typed netip.Addr, not user input.
  • The %[4]s slot replaces the blank line between [log] and [grpc] in the template. When dnsSection is non-empty it starts with \n and ends with \n, so the section boundary stays clean. When it's empty the blank line disappears — cosmetically different but perfectly valid TOML.
  • The boot graph wiring is clean: network.output flows into newBuildkitBootstartEmbedded, parallel to how registryHostMapping was already wired.

Tests: The two new test cases cover what matters — single and multiple nameservers produce the right TOML, and no [dns] section appears when the slice is nil. The existing "no hardcoded DNS" test was correctly reframed to check absence of [dns] rather than nameservers=.

One small cosmetic note: the generateConfig signature now takes four arguments where the first three were positional and the fourth is the DNS slice. The %[1]d/%[2]d/%[3]s/%[4]s explicit indexing in the format string makes the argument mapping clear and immune to accidental reordering. Good call.

Nothing blocking here — this is ready to merge.


🍪 full review note · comment /biscuit review to run biscuit again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant