Against C and Rust

Status: measured. The answer was 1.6× and is now a range.

docs/overflow-cost.md §4 named this as a measurement the repository owed and had not made: “It does not say the language is 40% slower than C. […] The comparison between cancho and C at equal semantics is a different measurement, and this repository has not made it.”

It is made now. The headline is that at equal semantics cancho is about 1.6× C and Rust, that Rust and C are within 4% of each other, and that the gap is therefore a backend gap rather than a language one.

Corrected by breadth, not by error. benchmarks-game.md added three more programs and the gap ranges from 1.17× to 2.58× across five. The two kernels here are inside that range and neither number below has changed; what was wrong was quoting their midpoint as though it were a constant. The range tracks how much of the run is in code Cranelift generated — 1.17× where the program is mostly inside malloc, 2.58× in a tight float loop — which is the same conclusion this document reaches, with a shape instead of a point.

§2’s own falsifier is now answered, and the whole range did collapse. docs/llvm-backend.md §7 measured mandelbrot.cho through --backend llvm against mandelbrot.c: 0.999×, not 1.786×. That was one point, re-measured while most of benches/ still couldn’t build on this backend at all. --backend llvm is the default now (#127), every kernel this document and benchmarks-game.md track builds on it, and benchmarks-game.md §8 re-ran all five of the wider set plus the two kernels below: §2’s own two numbers, re-run the same way (scripts/three.py --rounds 9, no --backend flag), move to mandelbrot 0.95× (two runs, both 0.95×) and sieve 0.93×–0.95× (two runs) — both now faster than C, not merely level with it. benchmarks-game.md’s own five kernels move less uniformly: the float-heavy one (spectral-norm) drops the most, the malloc-bound one barely moves, and one (fasta) moves the wrong way while staying faster than C — see that document’s §8 for the shape, not just the headline.

1. The discipline, because it is most of the work

Three rules, and each rules out a comparison that would have been easier and worthless.

The same algorithm, line for line — not the same task. Comparing examples/sort to GNU sort would measure decades of tuning and an external merge sort. benches/three/mandelbrot.{cho,c,rs} are the same loop written three times, with the same variables carried across iterations in the same order.

The same semantics. Rust’s release profile wraps on overflow; cancho traps. Reporting rustc -O against cancho would be reporting a semantic difference as a performance difference, so Rust is built both ways and the honest row is the one that also traps.

The same answer. Every build prints a checksum, and the harness refuses to report a time when they differ. That is not ceremony: an early draft had a fixed-point constant wrong in one language and the numbers looked fine.

$ python3 scripts/three.py

2. The numbers

linux-x86_64, Xeon @ 2.10GHz, minimum of seven runs, cc -O2 (gcc 13) and rustc -O (1.98).

Compute-bound: Mandelbrot, Q16.16 fixed point

All five compute 39,690,297.

Memory-bound: sieve of Eratosthenes

All three compute 6057.

2.1 Recounted: --backend llvm is the default now

Both tables above are --backend cranelift, whichever backend was the default the day they were measured. scripts/three.py passes no --backend flag to cancho build, so it has always measured whatever “the compiler” defaults to — Cranelift then, LLVM since #127. Re-run on a cloud container reporting the same CPU model (Intel(R) Xeon(R) Processor @ 2.10GHz, so not necessarily the same physical host or as quiet a one), two runs of scripts/three.py --rounds 9 each:

Both now faster than C, not merely closer to it — benchmarks-game.md §8 has the wider five-kernel set this pair sits inside, where the move is real but less uniform.

3. What the numbers say

1.6×, on both kernels, is the number to quote — for these two kernels. benchmarks-game.md §4 is why that sentence needed the qualifier it did not have. Not 1.0, and not the 40% figure from overflow-cost.md — that one measured trapping against wrapping within cancho, which is a different question and is why this document exists.

Rust is within 4% of C at equal semantics, and cancho is not. That is the load-bearing observation. Rust carries linear-ish ownership, a borrow checker, bounds checks and monomorphisation, and pays essentially nothing for them on this kernel. So the 1.6× is not the price of safety, ownership, or effect rows — those are erased at compile time here exactly as the README claims. It is Cranelift against LLVM, and it is the concrete version of the README’s “any larger gap early on is implementation maturity, not language design.”

That claim was an assertion when it was written. It now has a number attached and a falsifier: if an LLVM backend lands and the gap stays at 1.6×, the claim was wrong.

The falsifier is answered: the gap did not stay. docs/llvm-backend.md §7 landed a working (if partial) LLVM backend and measured this exact kernel, benches/three/mandelbrot.cho, against mandelbrot.c on the same host, same rounds, randomised (not merely interleaved) run order: cancho through --backend llvm is 0.999× C — indistinguishable — against cancho through --backend cranelift’s 1.786×, the same figure this document’s own 1.69× already said, within this host’s noise. The gap was Cranelift against LLVM, and closing it took switching backends, not touching the language. This is one kernel, not the whole range benchmarks-game.md widened to — most of benches/, including every program that range is drawn from, does not build on --backend llvm yet (docs/llvm-backend.md §7.3 has the inventory) — but it is the same kernel this document’s own headline number came from, and on it the claim holds exactly.

Rust beat C on the sieve, which is a warning about this whole document. 0.80× is not a language result; it is two implementations of one algorithm meeting different optimisers. The mandelbrot triple is the stronger evidence because the three sources are line-by-line translations with identical data layout, and the sieve pair is not quite that — Rust’s clearing loop is an iterator that LLVM turns into a memset and the C is a written-out loop. Treat the sieve row as a second opinion rather than a second measurement.

And the measurement floor is higher than it looks. The same cancho sieve, compiled from a version that returns its answer rather than printing it, ran 15% faster — a difference made entirely of code layout, the effect overflow-cost.md §3.3 found when padding an object moved a loop by a few bytes. Anything here under about 15% is not a result.

4. What the absence of float costs

This is the part that is not about speed.

cancho has no floating point (reach.md §2), so the Mandelbrot above is Q16.16 fixed point: a value is the real number times 65536, a product is (a * b) >> 16. The C and Rust versions were written the same way so the comparison would be fair — but a program that actually wanted this picture would have written double.

So here is that program, as the row cancho cannot enter:

4.1 The surprise: f64 is slower here

Double precision is 17% slower than Q16.16 integer arithmetic on this kernel, in both C and Rust, while doing slightly less work (the f64 runs total 39,684,084 iterations against fixed point’s 39,690,297).

Integer multiply and shift have better throughput on this processor than double multiply, and the kernel is three multiplies and four adds per iteration with no transcendental anywhere. So on this shape of work, not having floats is not a speed problem.

That is worth stating plainly because the intuition runs the other way, and because it narrows what the missing feature actually costs.

4.2 What it does cost: fifteen decimal digits

Ten orders of magnitude, and the picture shows it: the two runs disagree by 6,213 iterations out of 39.7 million, 0.0157%. Pixels near the boundary land on the wrong side because the grid coordinate itself is only accurate to 1.5 × 10⁻⁵.

For a benchmark that is nothing. For numerical work it is the whole subject:

• Deep zooms are impossible. Q16.16 holds values in ±32768 with 15 fractional bits. A Mandelbrot zoom past about 10⁴ has no bits left.
• Range and precision trade against each other, manually. Q16.16, Q32.32 and Q48.16 are three different types in a language with one integer, and choosing between them is the programmer’s arithmetic, checked by nothing. A float carries its exponent.
• Every standard numerical method assumes it. Condition numbers, convergence criteria, error bounds — the literature is written in floating point, and a fixed-point port of any of it is a new piece of analysis rather than a translation.

So the honest summary is that float is an expressiveness gap, not a performance gap, and reach.md §6’s row — “no design question that is known” — still holds. What this measurement adds is that adding it should be justified by what programs become writable, and not by a promise that they will get faster, because on evidence they will not.

5. What this does not measure

• Two kernels. Compute-bound and memory-bound, one each. Not a suite, and not startup time, code size, compile time or anything allocating in a loop.
• One machine, one afternoon, on the compilers that happened to be installed. scripts/three.py is committed so the table can be re-measured rather than believed.
• Nothing about safety. Every one of these programs is correct in all three languages. The interesting comparison — what it costs to write the wrong program — does not fit in a benchmark.
• No I/O. examples/base64 and examples/sort would have made better-looking programs to compare, and the comparison would have been against three different buffering strategies.

6. Open

7. The suite