Every developer who commits code knows the wait: the change is pushed, and somewhere a machine begins the slow business of deciding whether the work stands or falls. Before a single LLVM test compiles or checks anything, a test runner called lit parses and executes every line. That is pure overhead — the toll booth before the road. This summer, a student set out to lower the toll.
Prasoon Kumar, writing on the LLVM blog about his Google Summer of Code 2026 project, traces his interest in compilers to two courses at IIT Bombay — SysML and Advanced Computer Architecture — and to the realization of how much heavy lifting hides beneath a single line of code. He chose lit precisely because its problems were systems problems: process scheduling, IPC, syscall overhead, GIL contention. “Performance work here doesn’t stay contained to one project,” as he puts it — LLVM’s infrastructure sits beneath a large share of the toolchains people actually use. His mentors were Aiden Grossman, Paul Kirth, and Petr Hosek.
The burden of the RUN: line
lit is LLVM’s Integrated Tester. Its RUN: lines look like shell commands, but since a 2024 RFC made the arrangement the default everywhere, lit’s own internal shell — Python code, not a spawned shell process — parses and runs each one across the monorepo under ninja check-*. The option to spawn a real shell survives in the code but now raises an error unless forced, and is on track for removal. None of that parsing and executing compiles or checks anything by itself. It is, Kumar writes, the overhead sitting between a developer pushing a change and finding out whether it passed — and testing infrastructure, once it works well enough, tends to be left alone.
Hosek’s project idea flagged three suspects: builtin shell utilities like cat and diff spawning fresh Python interpreter subprocesses for every invocation line; locking and queueing overhead in the execution engine, multiprocessing.Pool, capping scaling on modern high-core-count machines; and Python 2 idioms lingering across lit’s execution paths. Before writing the proposal, Kumar ran lit under a debugger to find where its time actually went; the plan’s four parts — modernization, process overhead, internal-shell utilities, and profiling with Scalene — grew from that pass. A small script around hyperfine, later extended to measure peak RSS, supplied the timing numbers.
What the numbers said
The early weeks were spent reading every file in lit/ end to end: a suffix map dict replaced with an IntEnum, zip(range(len(x)), x) loops refactored to enumerate(x), two-argument super() calls modernized. Most were idiom changes with no measured speed difference. The real quarry was ShLexer, the shell lexer, which runs on every character of every RUN: line in every test file — and built each token with str += inside a character loop, continually reallocating intermediate strings, worst-case O(N²) on long tokens. Kumar switched the concatenation to list joins.
The result: CodeGen/X86 suite execution fell from 99.31 seconds to 94.96 seconds — 4.35 seconds, a 4.4 percent improvement. The Transforms suite went from 51.46 seconds to 50.89, a gain of 0.57 seconds or 1.1 percent.
The plan had also called for @dataclass(slots=True) on hot data objects created once per test, to drop the per-instance __dict__. Reality intervened: the nice syntax needs Python 3.10, and LLVM’s minimum supported Python is 3.8. Kumar shipped plain __slots__, which works on 3.8 and delivers the same benefit, and left a code comment pointing at GitHub issue #200531 so the switch happens when the minimum version rises. The pattern repeated all summer, he writes: ship the best version 3.8 supports, and name in a comment what should replace it.
Two quiet bugs
The deepest work concerned the execution engine itself, moved from multiprocessing.Pool to ProcessPoolExecutor. Along the way Kumar found that the old result-collection loop carried two defects. The --timeout was calculated once before the loop, so it never anchored to a single absolute deadline as intended. And test results were matched back to tests by list index rather than by identity — meaning out-of-order task completions could attribute a result to the wrong test.
There is something quietly satisfying about that second bug: the machine that delivers verdicts on everyone else’s code could, in a corner case, deliver someone else’s verdict. The fixes that matter most in infrastructure are rarely the ones anyone applauds. They are the seconds handed back, ten thousand times a day, to people who will never know whom to thank.