Originally IO used a version of SQLite that was translated into Go and then compiled as Go code. This allowed SQLite to be used without CGO, which allowed IO to be run in lightweight Alpine containers which include musl instead of libc.
But since IO is used alongside Envoy, this benefit is moot. Envoy depends on libc and the project has no interest in supporting musl. Since Envoy will always depend on libc, it seems reasonable for IO to also.
Surveying possible sources for a Go CGO-based wrapper, the landscape is complex:
- The “OG” implementation, crawshaw/sqlite, is unmaintained.
- The original implementation was forked into a “community-managed” one at go-llsqlite/crawshaw.
- The original author and several collaborators are currently maintaining tailscale/sqlite, which is essentially a publicly-visible internal library used by Tailscale.
This second implementation was the easiest to move to from zombiezen/go-sqlite, the library that IO originally used. However, I’m concerned about the long term stability of this repo, and after noticing the Tailscale repo, I decided to move to it. It has some significant API-level differences, but I prefer them:
- Read-only and read-write connections have separate types, making this more explicit in client code.
- Binary ([]byte) column values are easier to read.
- There is support for a tracer, which can be optionally added to view database usage details.
The Tailscale repo is clearly marked “work in progress” with no promises of support or stability To stabilize it for me, I’m maintaining a fork with some of my own experimental changes at agentio/sqlite. If you use it, fork your own version, and keep an eye on the Tailscale original.
Pros#
- This allows IO to directly use the C SQLite and reduces compatibility risks and performance overhead of the Go reimplementation.
- It’s fast! After some tuning, IO became performance-limited by the SQLite write-ahead logging of traffic events. For one simple test, throughput went from under 4000 requests per second using the Go-based library to over 25000.
Cons#
- With this, IO won’t be usable in Alpine containers.