How Differential Testing Exposed Type and Performance Bugs in Chuks v0.1.2

Building a programming language with both a bytecode VM and a native compiler creates an uncomfortable requirement: the same program has to mean the same thing in both places. Chuks has been working toward that guarantee release by release, first across the language itself and now across the standard library. Chuks compiles two ways: a bytecode VM you develop against, which starts instantly, and a native binary you ship, which runs with no runtime attached. Code like a script, ship like a binary. The previous release, v0.1.1, made those two modes agree on the language, every construct, every type rule, byte-for-byte. v0.1.2 does the same thing for the standard library, every module, checked call by call, driven directly from the committed listing of our public API surface. A method joins the verification the moment it joins the surface, so coverage can't quietly fall behind. Here's what that took, what it found, and the one fix that's the best illustration of why this work matters. The numbers 463 API entries now exercised directly by dual-mode differential tests, including three areas that had zero coverage before this release: std/net end to end, std/host, and the C FFI. 422 golden programs, green on both the VM and as a native binary. 1,569 differential cells across 21 suites. 11 preflight stages before any release, including a full build of every package in the repo and a cross-compile to all five supported targets. A machine-checked API stability gate: every public symbol in the stable tier is recorded in api/stable.txt , 682 entries, and a release build is diffed against it automatically. Removing or changing a stable symbol fails the release. This isn't a promise we're asking you to trust; it's a check that runs before we can ship. The bug that was both a correctness issue and a performance issue at once This is the one worth walking through in detail, because the fix illustrates the whole philosophy behind this release. Twelve array methods, .map, .filter, .reduce, .forEach, .find, and others were silently erasing their return type. const fruits: []string = ["apple", "banana"] var m = fruits.map((f: string): string => { return f.toUpperCase() }) const wrong: int = m[0] // this compiled. m was actually []any. Underneath, the native backend was boxing every element into a generic []interface{} before dispatching the callback. That's why the type was wrong, the result really was an untyped box, not a []string , and it's also why it was slow: boxing every element and dispatching through a generic interface has real overhead. The type bug and the performance bug were the same bug, because both came from the same design: a reflective, boxed code path instead of a specialized loop over []T.On the VM, this code worked fine and had the correct type, the VM's execution model didn't go through the same boxing path. So every test that ran during development, on the VM, passed. The failure only showed up on a compiled binary, at runtime, which is close to the worst place a type-safety bug can surface: the compiler said the program was correct, and it wasn't, and you find out on a device instead of at build time. How we found it Not from a bug report. We found it by changing our test methodology. The existing tests checked these methods by printing their output and comparing strings. That looks reasonable, except printing a []string looks identical whether the underlying value is a real []string or a boxed []any holding strings, the print path erases exactly the distinction that matters. So we rewrote the tests to consume the result as its declared type instead of printing it. That single methodology change surfaced three more bugs sitting under tests that had been green the whole time: .toSorted() was ignoring its comparator entirely. [10, 9, 2].toSorted((a, b) => a - b) returned 10, 2, 9 in a compiled binary, the old test used single-digit ascending integers, where character order and numeric order happen to agree, so it never caught this. .push() and .append() kept only their first argument. xs.push(1, 2, 3) left a one-element array in a native build and a three-element array on the VM silently, on the single most commonly used array method in the library. .reduce() had the same type erasure as .map(). The rule we took from this, which is now how we treat every bug a real program surfaces: A real consumer build is the compiler's best fuzzer, and every bug it finds gets converted into a generated test axis, not a single regression case. The fix, and why the speedup is trustworthy Twelve methods (map, filter, flatMap, forEach, find, findLast, findIndex, findLastIndex, every, some, reduce, toReversed) now compile to real loops over []T instead of boxing and dispatching reflectively. map's element type is now inferred from the callback's return, so the type-checker actually knows what comes out. On 300,000 elements, forEach and findIndex went from roughly 40ms to under 1ms. The detail that makes this number mean something for real code: the fast path used to depend on the callback capturing nothing, a trivial, non-capturing lambda. Most real callbacks aren't that; they close over a threshold to compare against, a counter to accumulate into, some outer variable. That's the shape of an ordinary callback, and it used to take the slow path unconditionally. The fix removes that dependency, so the speedup reaches the callbacks people actually write, not just the ones that make a clean benchmark. Errors say what went wrong, instead of a plausible wrong answer The same "fail loud, not silent" philosophy runs through the rest of the release: try { var cfg = json.parse(raw) } catch (e) { println("bad config: " + e.message) } json.parse and the base64 decoders now throw on malformed input instead of returning a value that reads as success. crypto.hmac and crypto.pbkdf2 reject an algorithm they don't implement rather than silently computing with SHA-256. buffer.toString rejects an encoding it doesn't implement. log.setLevel is now honored in a compiled binary, so a production build emits exactly the levels you configured, previously it could silently diverge from what the VM did.The buffer read family reports a short read instead of parsing zeroes: var n = frame.readInt32BE() // throws if fewer than 4 bytes remain Reading a map yields an optional, and it's free A key that isn't in a map has no value, so reading a typed map now gives you V?: var stock: map[string]int = {"apples": 3, "pears": 0} stock["kiwis"] == null // true stock["pears"] == null // false — a stored zero is a value, not an absence stock["kiwis"] ?? 0 // 0 Previously this was null on the VM and the zero value in a native build, so m[k] + 5 would print a number when shipped and throw a type mismatch in dev. Same source, two behaviors, the wrong way around. The migration is ?? default, and it costs nothing: that form compiles to a single lookup that never builds an optional, measured at the same speed as the old unchecked index over two million reads.Networking and the C FFI, now fully covered std/net gets end-to-end differential coverage for the first time: TCP round trips, a real TLS handshake, mutual TLS, peer certificate inspection, and the HTTP request surface including multipart uploads. upgradeToTLS now takes client TLS options, so STARTTLS can reach a server behind a private certificate authority, the common case for an internal mail relay or database: var conn = new TcpConnection("mail.internal", 587) conn.write("STARTTLS\r\n") conn.upgradeToTLS("mail.internal", { "ca": companyCA }) A CA that doesn't parse is refused explicitly (ca: no valid certificates found) rather than silently falling back to the public root store, so pinning a private CA actually stays pinned. Compiled HTTP servers also rebind immediately now; a deploy or container restart no longer races the previous socket's drain with bind: address already in use.std/chuksToC the largest module in the standard library at 96 entries, covering every call shape, pointer arithmetic, and every slice width, is now under dual-mode test as well. Failures raise a catchable error in a compiled binary exactly as they do on the VM: try { var lib = c.load("./native_lib/libmissing.so") } catch (e) { println("no native library: " + e.message) } And c.compile() builds a shared library in-process from C sources, so binding a C library needs no build system of its own:import { c } from "std/chuksToC" var lib = c.compile(["vendor/fast.c"], [], []) var addOne = lib.bindPI_I("add_one") println(addOne.call(nil, 41)) // 42 A stability guarantee you can check yourself api/stable.txt is committed to the repo, every exported symbol in the stable tier, with its full signature, 682 entries. A release gate stage compares what a build actually exports against that file and fails on a removal or a changed signature. Additions pass. This means the compatibility promise isn't a paragraph in a changelog you have to trust, it's a diff that runs before a release can ship. If you depend on a stable symbol, a release cannot take it away from you without the gate catching it first. Verification, and closing a hole in the harness itself The same audit discipline we applied to the standard library, we also applied to our own test infrastructure. A differential cell that failed to run as opposed to running and disagreeing, used to report clean rather than failing its suite. That's fixed: a cell that fails to run now fails its suite. This is the second time in two releases we've found and closed a hole in our own verification machinery, rather than just in the code it verifies. In v0.1.1 it was a skip condition that could mask a VM crash alongside a real regression. Different bug, same shape: not "did we get the answer wrong," but "would we actually notice if we did." Upgrading chuks upgrade Nothing requires a source change to compile. Three behaviors are stricter, and any code relying on the old ones was relying on a silent failure: json.parse, base64.decode, and base64.urlDecode throw on malformed input rather than returning null or an empty string. crypto.hmac, crypto.pbkdf2, and buffer.toString reject an algorithm or encoding they don't implement rather than falling back to a default. date.create reads its arguments as UTC in both modes, so the same call produces the same instant wherever it runs. upgradeToTLS gained an optional second parameter; existing calls are unaffected. The thing I'd want a reader to take from this release isn't the 40x. It's that the 40x and the type-safety fix were the same fix, found by a testing decision (consume as typed, not print) rather than a profiler, and that we now treat our own verification tooling as carefully as we treat the compiler, which is, I think, the actual job. If you want to see the full mechanism behind any of this, the differential harness, how api/stable.txt gates a release, the rest of what shipped, the original release notes are here: chuks.org/blog/chuks-v012-the-standard-library-release