Flintstack › Benchmarks

Benchmarks

Numbers from four corpora on one machine. Reproducible, not universal - switch corpus and the chart redraws from the data.

Four corpora, one machine. Pick a corpus and the chart and table redraw from the measured numbers; the data behind this page is benchmarks.json, documented on the data page. The method and the caveats are below the chart, and they are where the honest part lives.

Results

Corpus

The reference corpus: ~180k lines of C and C++, 9k of it generated, local fs backend only.

Wall-clock time for a full build of the selected corpus, median of seven runs. Same tree and toolchain within a corpus; only the state of the store differs between bars.

Exact values behind the chart. Speedup is relative to the cold run of this corpus.
Scenario	Store state	Median	p95	Speedup	Hit rate
cold	empty	708 s	721 s	1.0×	0.0 %
incremental	full, 12 % of units touched	202 s	219 s	3.5×	78.4 %
warm	full, nothing touched	69 s	74 s	10.3×	94.1 %

Method

Runs. Seven per scenario. We report the median and the 95th percentile; the mean is skewed by the first run after a reboot and is not shown.
Isolation. The page cache is dropped between runs. Without that the warm number is measuring RAM, not the cache, and falls far enough to be meaningless.
Machine. 8 vCPU, 32 GiB RAM, local NVMe, page cache dropped between runs. Store and build tree share the device, which is the pessimistic choice for the warm case.
Linking. Excluded from every corpus. The final link is one fixed step that the cache can hold but that dominates nothing; leaving it in would inflate every ratio equally.

Reading the corpora

The reference C/C++ corpus is the headline. The large-translation-unit corpus is the honest counterexample: forty multi-second compiles leave almost nothing for a cache to save, and the warm speedup collapses to roughly two. The monorepo figure includes a network restore over the objs backend, so its warm number is larger than a local store would give. The generated-heavy corpus plateaus below ninety per cent because a third of its sources are not reproducible and never match twice.

A corpus is not your build. The win scales with the number of cacheable invocations and with how reproducible their output is. If your tree looks like the large-TU corpus, expect its row, not the headline.

Variance

Cold runs are the steadiest, within about one per cent across repetitions, because they are compute-bound on an otherwise idle machine. Warm runs scatter by several per cent: at that point the build is almost pure I/O, and the slowest runs tend to land right after a prune handed the filesystem back a few gigabytes. A single warm run ten per cent slow is noise; a single cold run ten per cent slow means the machine changed.

Reproducing it

The harness lives in the tree under bench/. It fetches a pinned corpus, runs the three scenarios and prints a table shaped like the one above:

cd bench && ./run.sh --corpus cpp-2310 --jobs 8 --repeat 7 --drop-caches

It needs root for --drop-caches and refuses to run without it unless you pass --allow-warm-page-cache, which marks the output so it cannot be mistaken for these.

Last updated: 2026-09-29