Tuvix: Experimental hermetic build engine with Starlark

19 Mar 2022

DevOps  Rust 

Tuvix is an experiment I've been fiddling around with recently, building a hermetic build engine. By hermetic, I mean that it requires you to specify all the dependencies of a build as opposed to just getting them automatically from the OS. This has the advantage that you know what all the dependencies are and don't get randomly tripped up by something not existing in CI or disappearing at some point in the future. It also means you can potentially do really cool things like "have a cache of build artifacts that always perfectly knows about all the things that your build depends on" and then share those artifacts among people while avoiding the "but what if machine X has a different compiler?" problems.

It's called Tuvix both because I really like that Voyager episode and because after Nix and Guix I've obviously got to name my one something ending "ix". Those are the two biggest influences on things, although after getting as far as I have, I have much more sympathy than I used to for the Bazel folks. My original goal was "compile zsh" (as it's my shell of choice), which although it's a pretty simple task outside of this sort of structure, I really should have realised that it's harder than that. As it is, I've gotten to the point of having a hermetic build that starts the configure step, but then runs into problems with a lack of the compiler and I haven't gotten as far as "and now I hermetically build/bootstrap gcc" yet but I think I've learnt enough to stop the experiment.

What I've learned from all of those things is what a good hermetic package manager would look like, in large part by stealing from all the others

• It would look like Nix, in the sense of "support thousands of packages, let everything in" approach, but with an existing programming language as the DSL, not something new (e.g. Starlark)
• It would have an actual programming language for the config, like Guix does with Guile, except more readable (compare Guix's zsh build v.s the in-progress starlark one for Tuvix).
• It would be like Bazel, except that instead of defining things like py_binary as magic internal functions it should be written in Starlark as far down as possible, with the internal magic being as small as possible (adhering to the Lispy philosophy of little building blocks and then writing everything else in those). This is so these bits can actually get changed rapidly and easily, as opposed to requiring changes to the language core.

However, getting from here to "and here's a replacement for those" is a full time job for several people for the next few years, and given a lack of those people (heck, I don't have the spare time to expand this out further) I'm probably going to have to live with those suboptimal choices. I suspect one of these years I'm going to finally bite the bullet and try using Nix, if only because it's the most used tool in this space.

I've worked with a few new things along the way which are worth talking about:

• chroot is absolutely the way to do isolated builds. I've known of it before, and knew this was the right path from the outset, but hadn't used it before. Also, tools like "fakechroot" only get you so far and you really should just sudo and live with it. Also, you don't need the executable, just fs::chroot.
• overlayfs lets you do fun things like "here's the folders that contain the dependencies for this build, combine them in read-only mode along with a read/write 'current build' folder", and it just works (this is also how Docker does things)
• starlark (or to be more exact, the starlark-rust version, which annoyingly needs nightly Rust). It's basically like Python, except with almost all of the standard library stripped out so you can get a turing-complete configuration language with actually understandable and consistent syntax (looking at you Nix) and then bolt in a minimum set of external functions as opposed to having to lockdown the entire gigantic Python standard library.

Previously: Cavalcade: PostgreSQL-backed AMQP broker Next: Parallel for serial_test