What a program can reach

Status: measured, not argued.

The question this answers is the one every language gets asked and almost nobody answers honestly: can it do X? — where X is a REST API, a database client, a TLS connection, a thread pool.

The usual answer is a feature list. This document answers by building one of the X’s and reporting what happened. §1 is the program, §3 is the line it ran into, and §5 is the part of the story the effect row cannot tell.

1. A REST endpoint, and it works

examples/serve/ opens a TCP socket, binds a port, accepts a connection, routes the request line and answers with JSON:

$ cancho vcs fetch --lock examples/serve/net.lock \
      --store packages/net-sockets/.cancho-vcs -o /tmp/net-sockets
$ cancho build examples/serve/serve.cho /tmp/net-sockets/*.cho --std -o serve && ./serve 8080 &
$ curl -i http://127.0.0.1:8080/health
HTTP/1.1 200 OK
Content-Length: 11
Connection: close
Content-Type: application/json

{"ok":true}

The fetch line pulls in net.sockets (packages/net-sockets/), this repository’s first real cancho-vcs package (docs/package-system.md §6): the eight extern fns against libc it declares used to live in serve.cho itself, byte-for-byte duplicated in examples/results_stub/results_stub.cho, and now live in one published, locked, fetched file both import instead.

That is a real socket, a real accept, and a real client. The test harness makes the same request from Rust over loopback and checks the status line, the body and the declared length (an_http_server_written_in_cancho_answers_a_real_request), because a claim of this shape is worth exactly what it is tested with.

And it was worth it. The first version read the request with a single read and routed whatever arrived — which is wrong, because one read returns what has arrived rather than what was sent. On an idle machine it passed thirty runs in a row; under load it answered 404 to a request for /health about one run in six. Found while running the suite under load for an unrelated reason, and fixed with read_request (docs/porting.md §5). The socket is real, and so are the mistakes a real socket lets you make.

Nothing was added to the language for it. There is no socket type, no Net capability, no async runtime, no HTTP module in std. The program is 129 lines of code, of which eight are extern fn declarations against libc, and everything else is the slices, regions and capabilities that were already here.

Under valgrind, answering a live request:

total heap usage: 1 allocs, 1 frees, 65,536 bytes allocated
All heap blocks were freed -- no leaks are possible
ERROR SUMMARY: 0 errors from 0 contexts

One allocation, and it is the arena’s first chunk (ARENA_CHUNK). The request buffer, the response buffer and the sockaddr all come out of it, and it is released by leaving the region block. That is not tuning — it is what a program looks like when the allocation strategy is written in the source.

1.1 Why it works, stated as a rule

What decides whether a program is writable is not a feature list. It is whether the authority it needs has a name.

Sockets are libc. libc has a name — Ffi("libc") — and that capability has existed since M2. So the program exists, and it would have existed on the day the capability did. No feature had to be anticipated, because the thing that was designed was the authority, not the feature.

This cuts both ways, and §5 is the cut.

2. So the short answer

Corrected: opaque-pointers.md. §3.1’s own “everything that hands back an opaque handle is therefore out” is no longer true as stated. c_ptr is a real, distinct type — an opaque handle a foreign call returns or takes, checked for equality and nullness only, never dereferenced — which is exactly the shape §3.1.1 already accepted for an “integer-shaped pointer” like a file descriptor, made a first-class type instead of a convention. TLS is marked Partial rather than Yes for a reason that has nothing to do with the type system: cancho build only ever links libc, and a real TLS client needs -lssl -lcrypto, which the build pipeline has no way to ask for yet (opaque-pointers.md §5). A Postgres client is unaffected by this correction only because nothing has spent the same effort grounding libpq’s real signatures and writing its own accept fixture — the type-level gap PGconn * cited is the same one c_ptr closed.

Corrected again: docs/foreign-linking.md. The link-time gap above is closed too — -l/-L on build/run, and examples/tls_client/ is a real handshake and encrypted round trip against openssl s_server, on both backends. TLS moves from Partial to Yes above. A Postgres client stays No: the type-level and link-time gaps are both closed in general now, but nothing has yet spent the effort grounding libpq’s own signatures the way §1 did for OpenSSL’s.

Read down the “because” column. Two of the remaining three flat no’s share one sentence.

3. What is out of reach, and the one rule underneath it

3.1 A pointer does not come back

extern fn getenv[&f, &n](ffi: &f Ffi("libc"), name: &n [byte])
    -> [ffi("libc")] &n [byte];

error: `getenv` returns `&r1 [byte]`, which has no agreed layout across a
foreign boundary; a foreign result is `int`, `bool` or `c_ptr`, and a C
function that returns nothing is declared `int` and its result discarded

A foreign result is a scalar. Not because a pointer is hard to return, but because of what this language promises about the ones it already has: every reference here carries a region, the region says how long it is valid, and the checker enforces that. A pointer arriving from C carries none of that. Accepting one would mean either inventing a region for it — a lie the checker would then enforce — or admitting a reference the checker does not track, which is the undefined behaviour this whole language is built to define away.

So the rule is:

A value crosses into a cancho program only if the checker can say where it came from.

An int qualifies: it is a number, and a number’s provenance is nothing. A &r [byte] going out qualifies: the region is the caller’s and the callee is handed a pointer and a length that cannot disagree (strings.md §6). A pointer coming in does not, and that is the whole of the list in §2’s right-hand column.

Everything that hands back an opaque handle was therefore out: OpenSSL, libpq, libcurl, FILE *, dlopen. Not one of them for a reason of its own. Corrected: opaque-pointers.md §3 opens exactly one shape back in — a handle that is returned, passed around and compared for nullness, never dereferenced — which does not weaken this rule; it satisfies it the same way accepting a plain int already does, by claiming nothing the checker cannot back up.

3.1.1 And smuggling one does not help

The declaration is trusted — nothing checks a cancho signature against the C header — so malloc can be declared to return int and the compiler will believe it:

extern fn malloc[&f](ffi: &f Ffi("libc"), n: int) -> [ffi("libc")] int;

That compiles. What it buys is nothing:

let p = malloc(f, 64);
let s: &static [byte] = p;     // error: expected `&static [byte]`, found `int`

There is no cast, no from_raw, no transmute. A reference is produced by alloc, alloc_slice, borrow, a literal, an index or a subslice, and every one of those knows its region by construction. The soundness boundary is not the FFI check — it is the absence of a raw-pointer constructor. The FFI check is there so a programmer meets the rule at the declaration rather than three hours later.

An integer-shaped pointer can be handed straight back to C, which is how fork’s pid and a file descriptor already work. It just cannot be dereferenced here.

3.2 A struct does not cross either

bind wants a struct sockaddr_in. The program builds one:

let addr = alloc_slice[scratch](16, byte_of(0));
addr[0] = byte_of(2);                                 // AF_INET
addr[2] = byte_of(number / 256);                      // port, network order
addr[3] = byte_of(number - (number / 256) * 256);

Four lines, and they are the least defensible four lines in the program: nothing checks that 16 is the size, that 2 is AF_INET, or that the port is where byte 2 says it is. A #[repr(C)] equivalent would check all three.

It is worth being clear about what this costs and what it does not. It costs correctness at the boundary, which the programmer supplies by reading a header. It does not cost safety: the bytes are a slice, the slice has a length, bind is told that length, and a wrong guess is a kernel EINVAL rather than a read off the end of the buffer. That is the same trade strings.md §6 makes for write — the length C is told is the length the bounds check enforces — applied to a struct instead of a string.

3.3 Function values exist now; that still does not unlock threads

let h = g;

Corrected: function-values.md §4.2. This used to be refused outright; a captureless, non-generic, top-level function is a real value now, and h(...) calls through it. Still a scope decision rather than a soundness claim, and still load-bearing for the question people actually ask — it is still why threads are out, for a reason that turned out to be sharper than “there is nothing to pass”: threads.md §1 shows that pthread_create’s one void *arg is exactly §3.1’s own un-crossable pointer, so even with function values a callback that takes and returns void * is unreachable for the same reason malloc is. What function values unlock instead is threads.md’s own spawn/join, a primitive that never crosses the C ABI at all.

So examples/serve/ still answers one request at a time. A pre-forking server is expressible today with the same eight declarations plus fork and waitpid; a thread pool needs threads.md’s own primitive, proposed and not yet built.

3.4 A return crosses at a width this document never checked

§2 states the rule as “everything else crosses at cancho’s own widths,” and that is true and safe for a parameter: this backend writes a full 64-bit register, C reads the low bits its own narrower parameter type declares, and the bits above that are never read, so a caller writing more than the callee asked for costs nothing. A return is not the same shape, because the callee is the one who fills the register, and this document had never asked what a real int-returning C function leaves in the bits above 31.

The answer, found building examples/vsock/ and confirmed against real close/connect calls, checked byte for byte against equivalent C: sometimes zero, not sign-extended. close(999) (an invalid fd) on this platform’s libc compiles to a 32-bit store into the low half of the return register; x86-64’s own rule for writing a 32-bit register clears the upper 32 bits of its 64-bit parent as a side effect, so the register holds 0x00000000FFFFFFFF where a signed 64-bit reading expects 0xFFFFFFFFFFFFFFFF. Declaring the call’s return type as this backend’s own 64-bit int (abi.rs’s leaves, matching every other int in the program) and reading the full register back is exactly the bug this produces: 4294967295 where the real, POSIX-documented answer is -1.

Some extern declarations never hit this. labs returns C’s long, which really is 64 bits on both of this project’s targets, so reading the full register is correct there – confirmed by extern_fn_labs_computes_the_real_answer, which is exactly why that test never caught the bug a socket call’s own < 0 check did. read/write return ssize_t, also genuinely 64 bits (a transfer can exceed 2^31 bytes, however rarely in practice), so they must stay wide too. There is no way to tell these apart from the declaration’s int alone — close, connect, labs and read are four different real C widths (int, int, long, ssize_t, the first two the same by coincidence) spelled identically at the cancho boundary, and reading uniformly at 64 bits is silently right for two of them and silently wrong for the other two whenever the value is negative.

c_int names the common case explicitly, in return position only:

extern fn close[&f](ffi: &f Ffi("libc"), fd: int) -> [ffi("libc")] c_int;

Inside the program close(...) is still exactly an int — every assignment, comparison and arithmetic operation sees the same 64-bit value it always would. The only difference is at the call itself: the backend declares the foreign function’s return as a real 32-bit int (matching the platform C ABI on every target this project builds for) and sign-extends the result back to 64 bits itself, reading only the bits the ABI actually promised were meaningful rather than trusting whatever the callee happened to leave above them. c_int is not a general type — writing it anywhere but an extern fn’s return position is an unresolved name, the same refusal any other undeclared type gets.

Every existing socket declaration was migrated, and it was not cosmetic. examples/serve/’s and examples/collect/’s own bind/ accept/socket/close checks, and examples/fetch/’s and examples/report/’s own socket/close checks, all compare a foreign result against 0 with < somewhere in their error handling. Every one of those checks had this exact bug, silently, the whole time — untested only because the shipped test suite exercises their success paths, not a real bind/accept failure. read/write are unchanged: their real return is ssize_t, not int, and belongs on the wide side of this split.

tests/accept/foreign_narrow_return.cho checks access’s real -1 directly, on both backends; crates/cancho/tests/conformance/backends. rs’s the_two_backends_agree_on_a_narrow_foreign_return is the same claim through the CLI.

4. What was merely missing

Two things this program ran into that were not design at all, only absence. Both are fixed in the commit that adds it, which is the point of writing the program rather than reasoning about it.

\r was not an escape. HTTP’s line ending is CRLF. The literal could not say so, so the first draft assembled byte_of(13) into a buffer by hand — a protocol’s own separator, spelled as a number. strings.md §4 listed five escapes because nothing had asked for a sixth, which was true right up until something did. It is a byte with a spelling, not an encoding claim, so it belongs with \t rather than with \u.

The foreign-result refusal named a type the grammar refuses. It said a foreign result is int, bool, or () — and tuples.md §4 keeps () out of the source grammar deliberately, so a reader who took the advice was told a tuple has two components or more; there is no (). Two correct rules and a loop between them. The message now names what can be written and says what to do about a void function.

Neither is interesting. Both were found the same way, and neither would have been found by reading the documents.

4.1 And a third, which this document was credited with naming

examples/serve/ has four distinct non-zero exits and says nothing on any of them, because there was no standard error to say it on. Two other documents — file-handles.md §1.2 and ROADMAP.md — cite “reach.md §6’s standard-error gap” as the blocker for their own work.

There was no such row. §6 has five and none of them is this; the second citation was written from the first. A gap two documents think is recorded, and no document records, is worse than one nobody has noticed: it looks tracked.

standard-error.md is where it is written down, with what the absence cost measured, and closed — one more label on Io, no new capability, for standard-input.md §2’s reason. It reached this document’s programs too: base64 now says base64: invalid input where it used to exit 1 in silence. examples/serve/’s four remain, because a server’s diagnostics want somewhere to go, and that is a question about logging rather than about a stream.

5. Where narrowing stops

Here is the price of §1.1, and it is the part worth arguing about.

examples/serve/ listens on a port. Its authority report says:

performs
    args
    ffi("libc")
never touches
    the console
    the filesystem
    the heap

The row does not mention the network, and it is not wrong. The row is exact about what the program was granted: it calls a C library. It is silent about the network because libc is not an authority domain — it is every authority at once. Ffi("libc") grants sockets, and it grants them in the same breath as abs.

This is the one place the narrowing story runs out. Everywhere else it is exact in both directions: fs_read("/tmp") is a directory and not the filesystem, io_write is not io_read, an absent label is a proof (authority.md §2.2). At the foreign boundary the finest thing that can be said is a library’s name, and a library’s name is coarse.

5.1 Would a Net capability fix it?

No, and it would make the report worse. A Net capability would sit next to Fs and Io and mean this program may use the network — and examples/serve/ would still not hold one, because it reaches the network through libc, which is where the network is. Adding Net without taking sockets out of libc produces a program that opens a socket while declaring no net, which is a row that lies.

Taking sockets out of libc means builtins — socket, bind, listen, accept as prelude operations gated by Net(host), the way filesystem.md §2 does files. That is a real design and it is the right one if the network is a domain this language wants to mediate itself. It is also strictly larger than this document, it is the same shape as the filesystem’s, and the argument for it is not “the row should be finer” — it is “a host is a thing worth narrowing to”, which is exactly what lex-os’s perimeter says when it answers which host?.

Filed in §6 rather than answered here.

Promoted (#73): under-a-grant.md §5. The argument above frames this as a question of taste — whether a host is worth narrowing to — and that framing is now incomplete. Measured against lex-os’s real grant, two of its three dimensions cannot be decided without it: a cancho program that runs a network server reports args and ffi, so a grant saying network: None has nothing to refuse it with. The row is the prerequisite for cancho code running under a grant at all, not a refinement of the report.

And corrected (#74): net.md §1. “A host is a thing worth narrowing to” is the outbound question, and it describes neither side. examples/serve/ — the only network program here — declares bind, listen and accept and no connect: it is pure ingress. A lex-os grant’s network fields are network and egress, and both are outbound; there is no inbound notion in it at all.

Which is right, for a reason that decides the capability’s shape: outbound is the program’s choice and belongs in the type, and inbound is not — who reaches a listening port is routing’s answer, which is why lex-os enforces it with iptables on the tap device rather than in the grant. So Net has two axes: a host outbound, a port inbound, and nothing more on the inbound side.

5.2 What covers the difference today

The foreign symbol list, which exists for a different reason and turns out to answer this:

{
  "effects": ["args", "ffi"],
  "labels": [{ "name": "ffi", "argument": "libc" }],
  "foreign_symbols": ["accept", "bind", "close", "listen", "read",
                      "setsockopt", "socket", "write"]
}

A supervisor reading socket, bind, listen, accept knows it is being asked to run a server, and knows it without trusting a word the program says about itself: the list is what pass 2 emitted, which is what main reaches, which is what the binary can call (authority.md §2).

Corrected (#73): under-a-grant.md §3. The first half of that sentence stands and the second does not. The list is a proof about names — every foreign call needs a declaration, so it is exactly the set reachable from main — and a heuristic about domains, because the step from a name to a domain is the declaration, and the declaration is the program’s to write. Six lines make the point:

extern fn syscall[&f](ffi: &f Ffi("libc"), n: int, a: int, b: int, c: int)
    -> [ffi("libc")] int;
r = syscall(f, 41, 2, 1, 0);        // SYS_socket

It type-checks, opens a socket, and reports {"effects": ["ffi"], "foreign_symbols": ["syscall"]}. Against a cooperative program the heuristic is worth having. Against the threat model lex-os exists for, it is not a wall. The row is a grant question and the symbol list is a what will it actually do question, and at the foreign boundary the second is the one with the answer.

Built on (foreign-authority.md). The symbol list is now unbounded_by in the report, each symbol paired with the scope its declaration claims, and one program can hold several libraries. Ffi("libc") and a foreign function with no capability at all both used to be accepted; the second no longer is.

That is a report and not a type, and it should stay a report until §5.1 is answered. A row that said net today would be a row that could not be checked.

6. Open

7. The suite