Introducing NXD: Declarative Infrastructure Reconciliation for Nix-Authored State

NixOS solved the operating system. It did not solve what the operating system runs on.

My hosts are pure functions of pinned source. The Proxmox cluster underneath them was configured by clicking. The Headscale tailnet they join was enrolled by hand. The PBS appliance holding their backups had its retention policy stored in a web UI and nowhere else. I had perfect reproducibility above the hypervisor and none below it — and the seam between the two is exactly where my outages came from.

The usual answer is Terraform for the substrate, Ansible for the push, nixos-rebuild or Colmena for the switch. Three tools, three state models, and no single thing that knows a Headscale preauth key must exist before a VM boots that will try to use it. Ordering across that seam is left to the operator, which means it lives in a runbook, which means it is wrong.

NXD is a Rust reconciliation engine that removes the seam. One Nix-authored description of the whole environment — hypervisors, guests, tailnet, backups, and operating systems — evaluated into one canonical spec, planned as one dependency-ordered graph, and applied only against a digest you approved.

It is pre-release and under active development. It also runs my homelab.

The pipeline

Four properties make this different from a wrapper around shell commands:

Evaluation is offline and side-effect free. nxd.lib.evalConfiguration reads your modules and emits canonical JSON. It contacts no endpoint and decrypts no secret. Planning cannot be poisoned by the state of the thing being planned against.

Plans are artifacts with identities. A plan is written to disk and hashed. Approval binds to that hash. If the configuration changes between planning and applying, the digest no longer matches and the apply refuses. There is no window where you approve one thing and execute another.

Every action is risk-classified. IdentityCritical touches cryptographic identity — host keys, node registrations, credential bindings. ServiceImpacting mutates a running system — profile activation, reboots, service restarts. ReadOnly observes and asserts. You see the classification before you approve, not in a postmortem.

Verification is empirical. Success is not “the command exited 0.” It’s a re-read of live state asserting the target’s active system closure is the closure that was reviewed.

Ordering across providers is the point

The reason this is one graph rather than four tools is that the dependencies genuinely cross provider boundaries. Enrolling a new host:

1. Identity — resolve the host’s SOPS/age-encrypted SSH key material, pin the expected Ed25519 host key.
2. Overlay — mint a Headscale preauth key under tag:tagged-devices. This has to happen before the guest boots, because the guest consumes it at first activation.
3. Substrate — Proxmox VE creates the guest, attaches network adapters, seeds cloud-init.
4. OS — build the reviewed NixOS closure (on a builder, on the orchestrator, or on the target — see below), transfer over multiplexed SSH, activate.
5. Verify — assert the active closure and tailnet membership match desired state.

Step 2 before step 3 is not a convention someone wrote down. It’s an edge in the graph, and the scheduler enforces it.

Minting is also the reason enrollment is planned as its own stage: whether to mint is decided by reading the current persisted binding and matching it against live Headscale inventory, and minting mutates an input that the installing plan’s own planning reads. A single plan cannot honestly contain both. NXD splits them, then continues through both under one approval rather than prompting twice (ADR 0025).

Secrets never touch the store

The Nix store is world-readable. Any secret passed into a derivation is published to every user on the machine and permanently bound into a hash. So NXD does not put secrets in derivations.

Configuration carries binding references — secret/hosts/medo/tailscale-preauth-key — and nothing else. Canonical JSON carries the same references. At apply time, NXD resolves them through SOPS/age directly into process memory, or into staging files mode 0600 when a subprocess genuinely requires a path. Git holds ciphertext; the store holds references; plaintext exists only in the process that needs it, for as long as it needs it.

Transport is held to the same standard. nxd-transport is one shared SSH layer used by every provider, enforcing exact Ed25519 host key pinning with ControlMaster multiplexing on by default. That last part started as a correctness decision and turned into a performance one — before it was shared, twelve modules across four providers invoked SSH and exactly one multiplexed. The rest paid a full TCP and SSH handshake per remote command.

Providers

Providers are typed Rust crates, first-party and linked, speaking a common plugin protocol:

What it looks like

# Inspect the action graph before anything moves
$ nxd plan medo --source ".#nxdConfigurations.lamt"

# Apply a live system switch — digest approval required
$ nxd switch medo --source ".#nxdConfigurations.lamt"
evaluating selected target outputs...
evaluated selected target outputs in 4s
approval required for switch plan .nxd/plans/switch-4592a2f32894.plan.json
plan digest: sha256:4592a2f32894b465309c87f2052ab9b97ee19799898d1cfc5ec5b69cb446a3ea  (3 actions, 1 target)

PLAN ACTIONS:
  1. deployment-target/medo      build -> nixos-rebuild switch    [ServiceImpacting]
  2. secret/hosts/medo/ssh-host  stage SOPS decrypted key         [IdentityCritical]
  3. deployment-target/medo      verify active system closure     [ReadOnly]

✓ [medo] build placement configured=auto builder=deploy@utils
✓ [medo] realized closure /nix/store/jpz7lcmwjzjymdr38kxh428rwa2arr04-nixos-system-medo
✓ [medo] active system verified: desiredSystemPath == activeSystemPath (delta +0)

Re-enrolling a node whose endpoint has moved out of band:

nxd switch medo=203.0.113.48 --reenroll --source ".#nxdConfigurations.lamt"

Build placement is resolved per target rather than assumed. auto picks a builder matching the target’s system; a low-memory install that the orchestrator cannot realize itself evaluates the reviewed derivation on the orchestrator and realizes it in the target’s mounted store, validating that it produces the same reviewed output. A 2 GB node never has to evaluate nixpkgs.

Why it might belong in your stack

If you operate the infrastructure: one description of the environment instead of three that drift apart. Risk classification and digest approval mean you know what will change before it changes. The substrate stops being the undocumented part.

If you point coding agents at your infrastructure: every surface is structured. --format json on plan and verify, canonical JSON specs, typed provider contracts, explicit postconditions. An agent can inspect real state, propose a plan, and get empirical confirmation of convergence — rather than pattern-matching shell output and guessing. Fail-closed digest approval means an agent that gets it wrong produces a refused apply, not a broken cluster.

Get involved

NXD is open source and moving fast. Issues, ideas, and early adopters all welcome.

• GitHub — star or watch the repo: github.com/lamtt77/nxd
• Technical manual & architecture: nxd.lamhub.com/manual/next/
• Project site: nxd.lamhub.com

If you’ve felt the gap between a reproducible OS and an unreproducible substrate, that’s the gap this was built to close. I’d genuinely like to hear how it holds up against infrastructure that isn’t mine.