Debian Code Search drops its last cgo dependency

Debian Code Search now runs without any C code. Michael Stapelberg replaced a seven-year-old C compression library with a pure Go version built on Go's experimental SIMD package, and reports it matches the C original for speed. "I deleted the last cgo dependency in Debian Code Search," he wrote on his blog on September 6, 2026.

Debian Code Search is the service that lets anyone search 130 GiB of source code across every Debian package. It has been running since 2012.

What TurboPFor does and why C was there

The library in question is TurboPFor. It is an integer compression format, used inside an inverted index to store lists of document IDs compactly.

That job is unglamorous and performance-critical. A code search engine spends much of its time decoding those ID lists, so the decoder sets the speed of every query.

Go can call C, through a bridge called cgo. It works, but it costs. Every call crosses a boundary, cross-compiling gets harder, and the build now needs a C toolchain as well as a Go one. Stapelberg had carried that trade for seven years because the C version was faster.

What the Go rewrite achieved

The rewrite leans on simd/archsimd, an experimental package that arrived in Go 1.26. SIMD stands for single instruction, multiple data: one CPU instruction operating on a batch of values at once instead of one at a time.

Two figures from the post stand out. Positional popcount encoding, a bit-counting step in the format, ran about twice as fast with the SIMD path. Full blocks decoded roughly three times faster than the plain scalar Go version.

Measure	Result
Positional popcount encoding	About 2x faster with SIMD
Full-block throughput	About 3x faster than scalar Go
Against the original C library	Parity, encoder and decoder
cgo dependencies remaining	Zero

Parity is the important word. The Go version does not beat C. It draws with it, which is enough to make the C dependency not worth keeping.

The catch: it is still an experiment

This does not work out of the box. The package is gated behind a build flag. Stapelberg quotes the requirement directly. "Go 1.26 introduces a new experimental simd/archsimd package," he writes, "which can be enabled by setting the environment variable GOEXPERIMENT=simd at build time."

An experimental package can change shape between releases, and the API here is deliberately architecture-specific rather than portable. We covered Go 1.27 adding a portable simd package alongside the architecture-specific one, which is the direction this is heading, but neither is stable yet.

What this means for developers

The transferable result is that the cgo escape hatch is narrowing. Plenty of Go projects keep a C dependency for one hot loop, and accept a harder build in exchange. If SIMD in Go can reach parity with hand-written C on integer compression, that trade is worth re-examining in your own code.

Check what you are actually paying for cgo before you start. It is not only speed. It is cross-compilation, build toolchains in CI, static linking, and the debugging that follows a crash across the boundary. Stapelberg's win is a simpler build as much as a faster one.

Do not put GOEXPERIMENT=simd into a production build yet. Experimental means the package can move, and an architecture-specific API means your code is not portable by default. Prototype with it, measure your own numbers, and keep the C path until the API settles.

If you want the implementation rather than the summary, the Go port is published separately as goturbopfor. The service it feeds is open source under the Debian organisation. Reading a real SIMD kernel written in Go is currently a short list, and this is on it.