Register now for early access to the new Turso Cloud. Join the private beta

Carl Sverre ruined my day. And it was glorious

How a new tool from Antithesis helped us find 5 bugs in minutes and gave us the confidence to ship.

Glauber CostaGlauber Costa
Cover image for Carl Sverre ruined my day. And it was glorious

I am at this very moment attending a conference called BugBash: the premier conference on software reliability, hosted by our friends at Antithesis. It is my second year here, and it is a crowd I love. People who have been taking software reliability very seriously for years. I like most people here. Up until today, actually, I liked everyone here. Now, there's someone I severely dislike: Carl Sverre, their new "AI guy".

#The problem

For context, I need to tell you about a problem we have been facing: My company, Turso, has maintained for a couple of years a fork of SQLite called libSQL. libSQL can be used as a local database, but also accessed over the wire. We have a cloud product that does that, and taking advantage of how lightweight SQLite is, it allows you to create millions of individual, isolated databases to power your agents and more.

More recently, for the last year or so, seeing the growing number of limitations SQLite has and the difficulties of improving the fork significantly, we embarked on a journey to fully rewrite it: the Turso project (a journey, btw, that would not be possible without Antithesis finding a tremendous amount of bugs for us).

We also had a lot of learnings along the way: The core embedded database is something that needs to have an FFI interface to be exposed in the various languages (Typescript, Go, Python, etc). But a lot of platforms have serious trouble doing that. The situation in the Typescript world is particularly dire, with different platforms offering different runtimes, sometimes incompatible with each other, with different limitations, which makes loading those binaries very hard.

So in Turso, we decided to have two different packages. Focusing on Typescript, for simplicity, there is @tursodatabase/database, the actual database, and @tursodatabase/serverless, a pure-js implementation that will run absolutely anywhere.

The experience that we want is that these packages have to have the exact same API. They need to feel like one package: the user has to be able to switch between them by changing an import and nothing more.

#We've been reluctant

Turso is now at a point at which it implements over 95% of the SQLite language. We'd love to get to 100%, but we want to start getting people to use it! The problem is, most of our focus is on the database. The serverless driver exists, but because there is the alternative of just keeping using libSQL, which is production ready and works great for the networked use case, that is what we still recommend in our docs.

We do that because we have a lot more confidence in libSQL's network implementation. For all its warts it is battle-tested, and that is what most people will use to connect to our cloud product. But that also means that the alternative is either not talking too much about Turso, the local database, which we think is ready for people to start using, or ending up with a docs site like this:

Talking about just Turso and the Serverless driver would solve this, but now we risk making our Cloud users suffer with a package that is less mature.

#Carl Sverre, my new archenemy

Right after the keynote, comes Carl. We used to be friends until today, so I sit down and listen to what he has to say. And he is presenting two new libraries: Hegel and Bombadil (which seems cool too, but I will not be talking about it today).

The value proposition is clearly articulated: Hegel allows you to define properties about the system very easily, and with those properties defined, Hegel will automatically generate a series of automated tests that explore the realm of possibilities and enforce that property.

The moment he said that, a lightbulb went off in my head: "wait a second: 'the drivers have the same API and always produce the same results' is a property!" and a relatively easy one to encode. I had to take a meeting, so I went back up to my hotel room. I got there 10 minutes before the actual meeting time so I figured I'd just try that. I prompted Claude "Antithesis just launched this project called Hegel. Learn about it in their docs, then set it up so we can test the property that our drivers always have the same API and return the same thing"

After the meeting was over, I looked at the terminal, and there it was: a bug. Within minutes, using Hegel, I had found a bug. My jaw dropped so much that I felt ready to claim the throne of the Holy Roman Empire. I then proceeded to fix the bug. Then I decided to go understand the Hegel setup, and expand the testing set a bit to try and find more bugs (We were not, for example, using RETURN statements in the first batch, which can lead to some interesting serialization issues in SQL). Four more bugs. All in non-obvious edge cases.

At that moment, the world felt small. I could finally develop the confidence to ship my package, fix our docs, and get people on the path to adopt our new software, without sacrificing what was working. The problem with that is that between all the fixing, and prompting, and reading, I ended up staying in my room doing that almost the whole day, skipping the conference. I was so excited I just couldn't stop doing it.

Yes, Carl ended up saving me some 3 months of work and and pain. He might have helped my company be more successful. But I flew all the way here, wanted to mingle and meet people, and get to see the amazing talks that were lined up to happen. And I ended up spending the whole day so excited that I just couldn't stop fixing stuff.

Perhaps one day I'll end up forgiving Carl. But that day is not today.