Nix Execution Layer¶

Nix provides a reproducible, pinned runtime for Python, JavaScript, and bash stages. It is Noether's L1 — the layer below the stage store.

Why Nix¶

A stage is identified by its content hash. For that guarantee to mean anything, the runtime must also be reproducible: the same Python stage on two machines must produce the same output from the same input.

Nix gives us that via:

• Content-addressed derivations — every package is identified by the hash of its build recipe. Two Nix derivations with the same hash produce bit-for-bit identical outputs.
• No shared mutable state — packages live in /nix/store/<hash>-<name>, isolated from system libraries.
• Hermetic builds — Nix evaluation is hermetic at build time: network access is blocked, all inputs are declared explicitly. (This is distinct from the subprocess's network access at run time, which is unrestricted.)

How NixExecutor works¶

// crates/noether-engine/src/executor/nix.rs
pub struct NixConfig {
    pub timeout_secs: u64,       // wall-clock limit; process killed with SIGKILL on expiry
    pub max_output_bytes: usize, // stdout truncated to this length
    pub max_stderr_bytes: usize, // stderr truncated to this length
}

pub struct NixExecutor {
    store: Box<dyn StageStore>,
    nix_bin: PathBuf,
    config: NixConfig,
}

Execution flow per stage invocation:

1. Look up stage implementation (Python / Bash code string) in the store
2. Write to a temp file in /tmp/noether-<uuid>/
3. Build a minimal Nix shell (nix-shell --pure) with the required runtime packages
4. Spawn the process; pass input.json on stdin
5. A background thread waits for the child; the main thread calls recv_timeout(config.timeout_secs)
6. On timeout: kill -9 <pid>, return ExecutionError::TimedOut
7. On success: parse stdout as JSON; classify stderr for useful error messages

The Python wrapper receives the stage input as JSON on stdin and must write its output as JSON to stdout. Stderr is captured and used for error classification.

Error classification¶

NixExecutor inspects stderr to distinguish infrastructure failures from user code failures:

This classification is included in the ExecutionError::StageFailed message so agents and users can distinguish "Nix is broken" from "the Python code crashed".

Warmup¶

Cold Nix environments fetch Python/Node from cache.nixos.org, which takes 1-3 s. NixExecutor::warmup() pre-fetches the Python 3 runtime in a background thread at startup, so the first real stage invocation sees a warm cache:

NixExecutor::warmup(); // call once at CLI startup; returns immediately

The warmup runs a no-op nix-shell in the background and does not block the CLI.

Stage implementation languages¶

The stdlib uses InlineExecutor for all Pure Rust stages (zero overhead). NixExecutor is used for stages that need Python libraries (numpy, pandas, etc.).

Binary cache¶

Nix packages are fetched from cache.nixos.org on first use and cached in /nix/store. Subsequent runs of the same stage use the cache — startup overhead drops from ~2 s (cold fetch) to ~200 ms (warm).

In CI and production, a team can run a private Nix binary cache to share built derivations across machines.

Running the tests¶

# Unit tests (no Nix required)
cargo test -p noether-engine

# Integration tests (require Nix in PATH, warm cache recommended)
cargo test -p noether-engine -- --ignored

Integration tests are marked #[ignore] because they require downloading the Python 3 runtime on first run (~2 s cold) and would time out in CI.

Phase history¶