A Boundary-First Benchmark for Low-Latency Crypto Trading Systems

Search "low-latency crypto trading framework," and you will find a lot of microsecond claims. Almost none of them tell you where the stopwatch started and where it stopped. That omission is the whole game. "Sub-100µs" can mean the time from a network packet arriving at your NIC to an order acknowledgment coming back from the exchange, which would be extraordinary. It can also mean the time between two adjacent lines in a hot loop, which is a number you can manufacture at will. Both get written the same way in a README. So before publishing anything about my framework’s speed, I wrote down the boundaries first and measured second. The result: on a laptop-class Intel Core i7-1360P, across five runs of 900 post-warm-up order cycles each, run-level median latency was 121.1–135.4 µs and run-level p99 was 434.8–662.0 µs. Now, the part that matters more than the number. What the stopwatch actually covers The measured path is one synthetic top-of-book event turning into one locally generated order report, entirely inside a single host: Depth (md) → Strategy → OrderInput → TD (mock) → OrderReport Every timestamp in that chain is a journal frame generation time, read out of the event records after the run finished. Nothing was printed, formatted, or written to a trace file while the clock was running. What is inside the boundary: journal delivery between processes, callback dispatch, the strategy picking a side and a price, order construction, publication of the order-input frame, the trade process consuming it, and the generation of a local order report. What is outside: network sockets, exchange gateways, venue protocol encoding and decoding, matching-engine time, acknowledgments, fills, queue position, market impact, and profitability. All of them. The trade process is a native mock, not a real venue connector. That is a narrow claim on purpose. It measures the portion of the loop the operator actually controls and can actually optimize. If you are not colocated, your network path to the exchange will dominate this number by an order of magnitude, and no amount of shaving microseconds off the local path will change that. I would rather say that out loud than let a reader assume the 121 µs is an order-to-exchange figure. The architecture the number comes out of Three responsibilities live in three separate processes: market data (md), strategy, and trade execution (td), plus master and ledger services for coordination and state. They talk through a journal-based shared-memory event path, where every frame carries generation and trigger timestamps plus source and destination identity. The journal does double duty, and that is the design's actual point. During the run it is the transport. After the run it is the evidence. The same records the runtime used to execute are what the analyzer reads to reconstruct latency. There is no separate instrumentation layer that might be measuring something the runtime never did. Process isolation is an operational choice more than an architectural aesthetic. A flaky venue adapter can be restarted without stopping market data or touching the strategy process. A strategy crash does not corrupt trade connectivity. And because journal ordering makes clock stability load-bearing, the single-host topology used here deliberately keeps the whole measured path inside one system clock domain — multi-host clock skew is a real problem and I did not want it silently contaminating a first measurement. The strategy in the benchmark is native C++: the depth callback alternates buy and sell, takes the displayed best price on that side, builds one limit order, calls insert_order. Python is still in process bootstrap and module loading, but it is not in the measured callback. The framework exposes the same strategy interface through pybind11, so most people write strategies in Python and drop to C++ only where profiling says a callback sits on a critical path. A Python-path benchmark would answer a different and genuinely more practical question for most users, and I did not blend it into this result. That one is still to do. One more detail worth stating: the journal substrate builds on the open-source Kungfu runtime (Apache 2.0). The connectors, mock components, benchmark strategy, analyzer, and aggregation scripts are ours. I am also not claiming every journal operation is lock-free — establishing that would take a separate implementation audit, and I have not done one. The numbers Five independent runs. 5,000 synthetic depth events per run at a 300,000 ns interval, 1,000 orders emitted, first 100 matched cycles discarded as warm-up, 900 retained. 4,500 observations total. Processes pinned to specific logical CPUs. Journal-only tracing; benchmark CSV writing disabled. Run-level depth-to-local-order-report latency, in microseconds — the aggregation unit is the run, not the individual observation: Percentile Min Median Mean Max p50 121.1 121.3 124.9 135.4 p90 256.0 281.4 278.0 298.6 p99 434.8 447.3 485.9 662.0 The median is boringly stable across runs. The p99 is not — one run came in at 434.8 µs, another at 662.0 µs, a 50% spread on the same host with the same configuration. Run-level maxima ranged from 719.9 µs to 1,450.0 µs. This is precisely why reporting a single best run is a lie of omission. If I had run this five times and published the good one, the p99 would look 34% better and the number would be worthless to you. Splitting the path into its two stages: Stage p50 median p90 median p99 median Depth → order input 99.8 147.2 336.2 Order input → local report 22.3 131.4 187.7 End-to-end 121.3 281.4 447.3 (Stage percentiles are marginal quantiles, so they do not add up to the end-to-end row.) At the median, the first stage dominates: getting the depth event into the strategy and back out as an order request is most of the cost. In the tail, both stages contribute. Run 3 had a 1.334 ms maximum in the second stage and the worst end-to-end p99 — and I have to be honest that the current instrumentation cannot tell you why. Scheduler? Cache? A wake-up path? Some runtime mechanism? Attributing that excursion needs instrumentation I have not built yet. What I would push back on if someone else published this The paper has a threats-to-validity section, which in a blog post I would rather frame as: here is how I would attack this result if it were yours. Five runs is not enough for tail claims. 4,500 observations is an engineering baseline, not a statistical one. The artifact also reports p99.9, and with 900 retained observations per run that figure is determined almost entirely by a handful of largest values. I do not treat it as a stable statistic and neither should you. A serious version of this study predeclares at least 30 repetitions with longer runs and reports confidence intervals. The causal join is inferred, not propagated. Order inputs match order reports by order_id, which is solid. But depth frames match order inputs by nearest unused prior frame with the same symbol, side, and price. All 1,000 cycles per run resolved under the primary rule with no fallbacks — but that means the analyzer succeeded on a deterministic single-symbol workload, not that the join is provably unique. Under repeated prices, bursts, multiple symbols, or backlog it would not hold. A source event identifier needs to be carried through the whole order lifecycle before I run burst experiments. That is the single most important fix on the list. The workload is too polite. Constant-rate synthetic top-of-book messages have none of the properties that make real market data hard: clustered arrivals, size variation, venue-specific decoding, backlog after a burst, order-book reconstruction. Recorded replay and controlled burst datasets are needed. It measures a quiet system, not a busy one. The 300 µs input interval is a workload parameter, not a target. It works out to roughly 3,333 events per second, and since the median measured path is shorter than the input interval, this characterizes low-utilization latency. Saturation behavior is a different experiment. The environment is under-captured. Compiler flags, CPU governor, turbo and C-state settings, hybrid-core mapping, SMT sibling placement, interrupt affinity, and background load were not recorded in the published artifact. Results on other hardware could be higher or lower and I am not inferring a direction. There is a documentation inconsistency, and I am leaving it visible. The benchmark ran with a 100,000 ns MD spin window, which is what the result artifact and benchmark README record. A methodology document in the same snapshot said 50,000 ns. The stored result files are unaffected, but the repo needs fixing before the next measurement campaign. I mention it because a benchmark you cannot re-run from the published config is a benchmark you have to take on faith. It is not a comparison with anything. Not with Hummingbot, not with anything else. A fair cross-framework study needs pinned versions, equivalent strategy semantics, identical inputs and mock endpoints, disclosed tuning on both sides, and review from both communities. Nobody has done it. If someone from another framework wants to co-design one, I am genuinely interested. Reliability is not evaluated at all. Failure injection, recovery time, reconciliation after restart, multi-host clocks, container overhead, long-duration stability — none of it is in here. Process isolation and replayable journals are design requirements motivated by production experience, but this paper does not use production deployment as evidence for anything. The checklist, which is the real point If the number itself were the contribution, this would be a much shorter post. The methodology is what I would actually like to see more of. When you read anyone's latency claim, including mine: Where does the interval start and stop? If the endpoints are not named, there is no claim. Was tracing inside the measured path? Synchronous logging and CSV writes measure the logger. Are per-event records preserved and published? Summary statistics without underlying records cannot be audited. How many runs, and is run-to-run tail variability reported? One run is a demo. What is explicitly excluded? A paper that never says "this does not measure X" has usually measured something narrower than you think. Can you reproduce it from the published commit? Everything for this one is in the repo — source snapshot, launch scripts, journal analyzer, multi-run aggregation, plotting, and the stored results under scripts/benchmark/analysis/spin_100000_confirm. Artifact commit da9dd09839c6ce16ab73b1a7a11ae1b5ed4e9349: github.com/godzilla-foundation/godzilla-community Run it, break it, tell me where the methodology is weak. The next round should have propagated causal identifiers, 30+ repetitions, replayed market data with real burst structure, a separate Python-path result, and a controlled testnet experiment that finally puts a network on the other end.