enola./Blog/Dogfooding Enola

Engineering · Dogfooding

Dogfooding Enola: from a missed N+1 to a 2× faster TypeScript extractor

TL;DR

Fixing a missed N+1 query led me into Enola's TypeScript extractor. After simplifying the code, we profiled it. Much of the time went to searches that could never match. Removing that work cut extraction on a NestJS app from 1.9 seconds to 0.8 seconds.

It started with a real app

Enola indexed a production NestJS + Drizzle monorepo in 5.2 seconds and reported 51 performance risks, but not one database call inside a loop. That was the finding I expected most.

A plain scan for await inside loops turned up 136 candidates, and reading the code confirmed real N+1s. The worst was a user-facing endpoint that ran about nine queries per item.

Enola's own facts showed why it missed them:

Extending the extractors

After four changes, Enola flagged 43 high-severity cases of I/O inside loops. These included every N+1 query we had found by reading the code.

Two coverage-report fixes came out of the same session. A backend whose calls all resolve to its own routes is no longer labelled a "coverage gap", and enola coverage --unresolved now lists each unresolved call site with file and line.

Measure (NestJS + Drizzle app)BeforeAfter
Functions recognised as doing I/O 47 511
Functions that call I/O through other functions 231 1,320
High-severity call-in-loop findings 0 43
Mobile API calls linked to backend routes 158 / 170 170 / 170
Backend coverage classification coverage_gap (false alarm) connected

Dogfooding: Enola on Enola

Adding those features meant passing one more parameter through functions that already took 13. Before adding more, I wanted to see where the complexity was. We ran Enola on its own repository.

Enola indexed itself in about 11 seconds. Its TypeScript extractor held 460 functions with a total cyclomatic complexity of about 4,000, and ten functions at 40 or above. The worst four sat on the hot path every TypeScript file goes through:

Each new feature meant editing one of these functions. Reviewing another change inside a function with cyclomatic complexity of 78 was getting difficult.

Refactoring with Enola as the safety net

The refactor ran in three steps, and after each one the extractor produced byte-identical facts on five real TypeScript repositories, about 52,000 facts in all.

The check was simple: extract each repository before and after, sort the output, compare the bytes. A refactor that changed one relation on one symbol in those repositories would have failed it.

FunctionComplexity beforeAfter
extractNode7813
tsBodyWalker.walk7712
Extract494
extractFile477

No new function has a complexity score above 13. The total for the package barely changed: the same decisions were still there, spread across smaller functions. Each piece can now be read, tested and reviewed on its own. Extraction time stayed about the same.

Making it fast

A CPU profile of extraction on the largest app showed that most of the time went to searches in files that could never match. On a NestJS app with 20,000 facts, extraction fell from 1.9 s to 0.8 s.

For each performance change, we compared the extracted facts before and after. The initial TypeScript checks matched byte for byte. The full benchmark below extended that comparison to more repositories and caught a bug those checks had missed.

TypeScript extraction became 1.3–3.6× faster across 16 repositories. ClickHouse improved 3.6×, Excalidraw 2.7×, Storybook 2.5×, and the NestJS plus Drizzle app 2.4×. Output matched byte for byte on every repository.
TypeScript extraction before and after the performance changes, best of three runs. Fact counts refer to the TypeScript extractor; ClickHouse is mostly C++. These are extraction timings, not full-index timings. Select the chart to view it at full size.

The allocator change also helps languages we did not touch: on the Codex repository, the Rust extractor dropped from 8.1 s to 6.9 s.

Running the full benchmark

I wanted to check the whole change, including the code outside TypeScript. We ran the full benchmark on 124 public repositories, from small libraries to the Linux kernel. The suite checks performance findings against human labels and measures indexing time. TypeScript is the largest group, with Angular, Vue and Svelte projects among them, and the rest are C and C++, C#, Dart, Go, Java, Kotlin, PHP, Python, Ruby, Rust, Scala and Swift. Every part compared a build of main with the new build.

  1. Grading: the performance findings, checked against the labels.
  2. Timing: a cold full snapshot of all 124, alternating the two builds, with a 15-minute cap per run. None hit it.
  3. Re-timing: the 16 largest repositories, again, with a warm-up and the median of repeated runs.

What changed in the findings

The benchmark reported 51,355 findings on main and 51,353 on the new build. We reviewed every change. Eight repositories changed, all because of the TypeScript work: 14 findings are new at high severity because the I/O is now confirmed, 3 are new at medium, and 19 false positives are gone.

Repositorymain → newWhat changed
directus420 → 40614 path.join and Array.join findings dropped
payload744 → 7476 new, 3 dropped
spartacus (Angular)240 → 2444 new, from services calling through facade interfaces
drizzle-orm324 → 3273 new, confirmed I/O
n8n2,797 → 2,7992 new
gauzy (Angular)1,006 → 1,0042 dropped, Array.reduce calls that do no I/O
angular950 → 9511 new at high severity: a stat call in a loop in the compiler's file-system code
thingsboard1,447 → 1,4481 new, in its Angular front end

Measuring the speedup

A single cold run of each of the 124 repositories took 1,698 s with main and 1,578 s with the new build (1.08×). Single runs turned out to be unreliable for large repositories, so we re-timed the 16 largest with a warm-up run and the median of two or three runs per build. None was slower on the new build.

Full-snapshot speedups across 16 large repositories range from 1.01× to 1.65×, with a combined speedup of 1.08×.
Full snapshots, measured after a warm-up using the median of two or three runs per build. Select the chart to view it at full size.
RepositorymainNew buildSpeedup
Peertube (Angular)47.0 s28.5 s1.65×
Storybook (TypeScript)6.3 s4.4 s1.44×
Nextcloud server (PHP)17.6 s12.8 s1.37×
n8n (TypeScript)46.1 s36.5 s1.27×
Grafana (Go)29.4 s24.0 s1.22×
Discourse (Ruby)40.6 s34.8 s1.17×
ClickHouse (C++)90.8 s83.8 s1.08×
.NET runtime (C#)90.5 s85.0 s1.06×
Roslyn (C#)32.5 s30.6 s1.06×
rust-lang/rust36.8 s34.8 s1.06×
Shopware (PHP)73.9 s70.1 s1.05×
Spark (Scala)18.3 s17.5 s1.05×
Dart SDK73.5 s70.7 s1.04×
GitLab (Ruby)242.5 s234.5 s1.03×
Linux kernel (C)232.0 s228.4 s1.02×
OpenProject (Ruby)36.9 s36.4 s1.01×
Sum of the 161,115 s1,033 s1.08×

On ClickHouse, TypeScript extraction is 3.6× faster, but it handles only about 2,000 facts in a mostly C++ repository. The full snapshot improved from 90.8 s to 83.8 s, about 8%.

Checking the extracted facts

We also compared the facts behind the findings. The counts matched between the two builds on every repository except six that contain TypeScript: Angular's dashboard, NestJS, tRPC, TypeORM, Zod and Deno. Five differences came from the new detection, though TypeORM also had noise from its test fixtures. Deno was different: our speedup had dropped two dependency facts. We found and fixed that bug.

Bugs we introduced

Alongside the Deno bug, our benchmark and CI checks caught three more mistakes:

  1. A false-positive filter was too broad. It treated every node: module as non-I/O, so node:fs calls such as writeFile stopped counting, and it skipped verbs like delete, which cost a finding the labels mark as correct. The labelled run caught it. We narrowed the filter to the names that caused false positives and added a test.
  2. The interface-member handling we added for TypeScript lives in code shared by every language, so Scala traits picked it up and gained three facts each. Comparing facts across languages caught it, and we made the behaviour opt-in, set only by the TypeScript extractor.
  3. The CI job that runs Enola on its own repository failed on a layer violation in our new code: the unresolved-call-site listing imported a linker package only to read two property names. We moved the names into the shared facts package, and the same run flagged, as advisory, a loop in our ORM detection that re-read each workspace package.json three times. It now stops as soon as all three ORMs are found.

These checks caught mistakes in our own code, including changes outside TypeScript that we had not intended. That gives me more confidence in the release.

Six functions in Ember, GraphQL and Vue support still have complexity scores of 40 or more. Those are next.

enola is open source under Apache 2.0. Free, local, no data leaves your machine.

shell
curl -fsSL https://raw.githubusercontent.com/enola-labs/enola/main/install.sh | sh