How Enola is measured
Five questions decide whether Enola's graph and its verdicts can be believed. Each is measured on public repositories, with one binary and scripts you can re-run, and where a result is unflattering it is still published.
Every benchmark run updates the full results, tables and method in docs/BENCHMARKS.md. This page shows the headline of each.
1 of 5
Every repository is indexed three times, once cold and twice warm, and the snapshot ID and the hash of the fact file are compared across all three. Running cold and warm is the point: a cached run and a from-scratch run must agree.
The snapshot ID is a hash of the facts, the version and the config, so two runs on the same commit are provably the same graph. A diff between snapshots means something only because of this.
Full, current results →2 of 5
Pin a baseline on a real repository and move it through four states. Only the injected cycle may fail, and it must be reported as exactly one regression, however many findings the repository already carried. The runs enforce --fail-on=cycles: Enola fails nothing unless a policy names it.
3 of 5
Committed fixtures pair services across languages (TypeScript, Go, Python, PHP, Dart, Protobuf, OpenAPI), each with a known answer. Enola must link each call to the code that serves it and report every call it could not link, instead of hiding it or guessing.
The benchmark also runs real public clusters, including ones where Enola resolves little and says so: a service whose calls it could not follow is classified as a coverage gap, never as isolated.
Full, current results →4 of 5
Every sweep indexes the whole public corpus, from small applications to the Linux kernel, GitLab and dotnet/runtime, recording time and peak memory per repository. A graph the other questions rely on has to be buildable on real code first.
Speed is recorded, not claimed: Enola is not benchmarked on speed against other tools.
Full, current results →5 of 5
One coding task, given to Claude Code three ways: bare, with Enola's MCP server and instruction, and with the hooks that check the change when the agent stops. The obvious implementation closes a dependency cycle that compiles and passes every test.
Nine trials is a demonstration that the mechanism works end to end, not a measured effect size. No arm asked for the comparison itself; the hooks ran it.
Full, current results →"The regression wassrc/domain ↔ src/store: my first attempt made the domainimport { listOrders }from the store, which closed a loop with the store's pre-existingimport type { Order }from the domain. I removed the offending edge."Unprompted, from a transcript with the hooks installed
Every page like this one shows you the measurements that survived. Here are the ones that did not, and why.
Before the results above, we ran 510 headless agent sessions across three separate benchmark designs, and almost none of it survived scrutiny. The oracles we chose (gopls, tsc, go build) could only grade statically resolvable questions inside a single repository, which is the same class of question a text search answers well. Requiring a mechanical oracle had silently selected the questions our baseline was already good at. Two of six answer keys turned out to be wrong, and in both contested cases the agents had been more correct than the key grading them. The one arm that genuinely exercised the tool was an arm we invented, which stripped the agent of grep and shell, and which describes a configuration nobody runs.
What survived was a defect detector: the exercise found a real bug in Enola's own config parser that had gone undetected across a 72-repository corpus, because it was invisible on any repository with a single package.
The write-up names all six ways the harness lied, including a control group that silently inherited a running instance of the tool under test.