In this week’s Yak shaving: Trying to write a Terraform Provider in Rust.
I had an idea for a Terraform Provider, and given that all of Terraform is written in Go, it’s standard advice for writing Providers (which are plugins that provide a collection of resources e.g say AWS for things like EC2 instances) is to write them in Go. That they’ve got 2 SDKs both entirely done for Go and they don’t support anything else means this is a bad idea. However, I’m not very fond of Go (I’ll write patches for other people’s Go but I try and avoid writing any of it in my own projects), and so figured it can’t be that hard right?
This is half-correct. The core protocol is mostly gRPC since Terraform 0.12, and that’s old enough that I don’t care much for anything further back. There’s some subtleties in that “mostly”, but I’ve gotten something working.
Firstly there’s the handshake on stdout. Providers are booted as subprocesses of Terraform, and then need to output some magic to tell Terraform where the gRPC endpoint is. Luckily the spec for this is documented, although it’s not totally accurate (e.g. the Health Checking Service isn’t used). There’s also the PLUGIN_MIN_PORT/PLUGIN_MAX_PORT environment variables to pay attention to, and then the fun of PLUGIN_CLIENT_CERT which is used as part of the mTLS setup. Basically, your process gets an environment variable containing a client certificate (which you need to implicitly trust), and the last part of the handshake needs to be your server cert, and then everyone can talk. Except that they picked the really lovely choice of ECDSA SHA-512, which isn’t supported by ring (mostly because it’s a rarely used option) so I had to make my own client verifier that mostly just goes “sure, that looks about right”, but then we’re in business.
The protocol is fairly large so I’ve mostly just got a lot of unimplemented!() blocks and a few bits of real code when Terraform actually asked for something. One of the interesting choices is that there’s a type field of type bytes, which is always a good start. Turns out that under the hood Terraform uses a library called cty and if you dig around in there, eventually you find out the magic bytes are mostly just strings saying what something is. There’s also the entertainment of legacy_type_system - gotta love a doc comment saying DO NOT USE THIS.
Right now, this only implements the bits necessary to get terraform plan to run (including a basic integration test for that), and other things e.g. apply don’t work, but that’s a matter of “I didn’t need them yet”. I’ve also uploaded the provider to the registry (mostly to check it would work with that) so if you really feel like using this, you can (well, provided you’re on a x64 Linux machine, as I haven’t compiled it for anything else yet).
Next steps will be to eventually (although I’ve got a few other more important things to work on) build this out into the Provider I originally wanted to build before I started on this yak, and then trying to split out the “generic Provider” bits from the “specific Provider” bits to make a proper library for this sort of thing, as well as testing with a few more versions of Terraform, but that’s all a bit down the list now and I’m satisfied with where this has gotten to.
Comments
With an account on the Fediverse or Mastodon, you can respond to this post. Since Mastodon is decentralized, you can use your existing account hosted by another Mastodon server or compatible platform if you don't have an account on this one. Known non-private replies are displayed below.