blog-static/content/blog/blog_with_nix.md

---
title: "Declaratively Deploying Multiple Blog Versions with NixOS and Flakes"
date: 2022-04-10T00:24:58-07:00
tags: ["Website", "Nix"]
---

### Prologue

You can skip this section if you'd like.

For the last few days, I've been stuck inside of my room due to some kind of cold or flu, which
or
{{< sidenote "right" "pcr-note" "may or may not be COVID™." >}}
The results of the PCR test are pending at the time of writing.
{{< /sidenote >}}
In seeming correspondence with the progression of my cold, a thought occurred in the back of my mind:
"_Your blog deployment is kind of a mess_". On the first day, when I felt only a small tingling in
my throat, I waved that thought away pretty easily. On the second day, feeling unwell and staying
in bed, I couldn't help but start to look up Nix documentation. And finally, on the third day,
between coughing fits and overconsumption of oral analgesic, I got to work.

In short, this post is the closest thing I've written to a fever dream.

### The Constraints

I run several versions of this site. The first is, of course, the "production" version, hosted
at the time of writing on `danilafe.com` and containing the articles that I would like to share
with the world. The second is a version of this site on which drafts are displayed - this
way, I can share posts with my friends before they are published, get feedback, and even just
re-read what I wrote from any device that has an internet connection. The third is the Russian
version of my blog. It's rather empty, because translation is hard work, so it only exists
so far as another "draft" website.

Currently, only my main site is behind HTTPS. However, I would like for it to be possible to
adjust this, and possibly even switch my hosts without changing any of the code that actually
builds my blog.

I wanted to be able to represent all of this complexity in my NixOS configuration file, and
that's what this post is about!

### Why Flakes
I decided to use Nix flakes to manage my configuration. But what is it that made
me do so? Well, two things:

* __Adding custom packages__. The Nix code for my blog provides a package / derivation for each
  version of my website, and I want to use these packages in my `configuration.nix`. Adding
  custom packages is typically done using
  overlays; however, how should my system configuration get my overlay Nix expression? I would
  like to be able to separate my build-the-blog code from my describe-the-server code, and
  so I need a clean way to let my system access the former from the latter.
  Flakes solve this issue by letting me specify a blog flake, and pull it in as one
  of the system configuration's inputs.
* __Versioning__. My process for deploying new versions of the site prior to flakes boiled down to fetching
  the latest commit from the `master` branch of my blog repository, and updating the `default.nix`
  file with the corresponding hash. This way, I could reliably fetch the version of my site that
  I wanted published. Flakes do the same thing: the `flake.lock` file
  contains the hashes of the Git-based dependencies of a flake, and thus prevents builds from
  accidentally pulling in something else. However, unlike my approach, which relies on custom
  scripts and extra tools such as `jq`, the locking mechanism used by flakes is provided with
  standard Nix tooling. Using flakes also guarantees that my build process won't break with
  updates to Hugo or Ruby, since the `nixpkgs` version is stored in `flake.lock`, too.

### The Final Result
Here's the relevant section of my configuration:

{{< codelines "Nix" "server-config/configuration.nix" 42 59 >}}

I really like how this turned out for three reasons. First,
it's very clear from the configuration what I want from my server:
three virtual hosts, one with HTTPS, one with drafts, and one with drafts and
_in Russian_. Second, there's plenty of code reuse. I'm using two builder functions,
`english` and `russian`, but under the hood, the exact same code is being
used to run Hugo and perform all the necessary post-processing. Finally, all of this can be
used pretty much immediately given my blog flake, which reduces the amount of glue
code I have to write.

### Getting There
#### A Derivation Builder
As I mentioned earlier, I need to generate multiple versions of my blog. All of these
use pretty much the same build process -- run Hugo on the Markdown files, then do some
post-processing (in particular, convert the LaTeX in the resulting pages into MathML
and nice-looking HTML). I didn't want to write this logic multiple times, so I settled
for a function that takes some settings, and returns a derivation:

{{< codelines "Nix" "blog-static-flake/lib.nix" 6 21 >}}

There are a few things here:

* On line 7, the settings `src`, `ssl`, and `host` are inherited into the derivation.
  The `src` setting provides a handle on the source code of the blog. I haven't had
  much time to test and fine-tune the changes enabling multi-language support on the site,
  so they reside on a separate branch. It's up to the caller to specify which version of the
  source code should be used for building. The `host` and `ssl` settings are interesting
  because __they don't actually matter for the derivation itself__ -- they just aren't used
  in the builder. However, attributes given to a derivation are accessible from "outside",
  and these settings will play a role later.
* Lines 10 through 14 deal with setting the base URL of the site. Hugo
  does not know how to interpret
  the `--baseURL` option when a blog has multiple languages. What this means is that in the end,
  it is impossible to configure the base URL used in links from the command line.
  I need to apply some manual changes to the configuration file. It's necessary to adjust
  the base URL because each version of my website is hosted in a different place: the default (English)
  website is hosted on `danilafe.com`, the version with drafts on `drafts.danilafe.com`, and so on.
  However, the configuration file only knows one base URL per language, and so it _doesn't_ know
  when or when not to use the `drafts.` prefix. The `urlSub` variable is used in the builder.
* On line 15, the `publicPath` variable is set; while single-language Hugo puts all the generated
  HTML into the `public` folder, the multi-language configuration places it into `public/[language-code]`.
  Thus, depending on the configuration, the builder needs to look in a different place for final output.

This new `website` function is general enough to represent all my blog versions, but it's too low-level.
Do I really want to specify the `publicPath` each time I want to describe a version of the site?
What about `settings.replaceUrl`, or the source code? Just like I would in any garden variety language,
I defined two helper functions:

{{< codelines "Nix" "blog-static-flake/lib.nix" 25 48 >}}

Both of these simply make a call to the `website` function (and thus return derivations), but they
make some decisions for the caller, and provide a nicer interface by allowing attributes to be omitted.
Specifically, by default, a site version is assumed to be HTTP-only, and to contain non-draft articles.
Furthermore, since each function corresponds to a language, there's no need for the caller to provide
a blog version, and thus also the output `path`, or even to specify the "from" part of `replaceUrl`.
The `wrapHost` function, not included in the snippet, simply adds `http` or `https` to the `host`
parameter, which does not otherwise include this information. These functions can now be called
to describe different versions of my site:

```Nix
# Default version, hosted on the main site and using HTTPS
english {
    ssl = true;
    host = "danilafe.com";
}

# English draft version, hosted on draft domain and not using HTTPS.
english {
    drafts = true;
    host = "drafts.danilafe.com";
}

# Russian draft version, hosted on draft (russian) domain, and not using HTTPS.
russian {
    drafts = true;
    host = "drafts.ru.danilafe.com";
}
```

#### Configuring Nginx
The above functions are already a pretty big win (in my opinion) when it comes to
describing my blog. However, by themselves, they aren't quite enough to clean
up my system configuration: for each of these blog versions, I'd need to add
an Nginx `virtualHosts` entry where I'd pass in the corresponding host (like `danilafe.com` or
`drafts.danilafe.com`), configure SSL, and so on. At one point, too, all paths
in `/var` were by default mounted as read-only by NixOS, which meant that it was necessary
to tell `systemd` that `/var/www/challenges` should be writeable so that the SSL
certificate for the site could be properly renewed. Overall, this was a lot of detail
that I didn't want front-and-center in my server configuration.

However, with the additional "ghost" attributes, my derivations already contain most
of the information required to configure Nginx. The virtual host, for instance,
is the same as `replaceUrl.to` (since I'd want the Nginx virtual host for a blog
version to handle links within that version). The `ssl` ghost parameter corresponds
precisely to whether or not a virtual host will need SSL (and thus ACME, and thus
the `systemd` setting). For each derivation built using `website`, I can access
the attributes like `ssl` or `host` to generate the corresponding piece of the Nginx configuration.

To make this _really_ nice, I wanted all of this to be "just another section of my
configuration file". That is, I wanted to control my site deployment via regular
old attributes in `configuration.nix`. To this end, I needed a module. Xe recently
[wrote about NixOS modules in flakes](https://christine.website/blog/nix-flakes-3-2022-04-07),
and what I do here is very similar. In essence, a module has two bits:

* The _options_, which specify what kind of attributes this module understands.
  The most common option is `enable`, which tells a module that it should apply
  its configuration changes.
* The _configuration_, which consists of the various system settings that this module
  will itself set. These typically depend on the options.

In short, a module describes the sort of options it will accept, and then provides
a way to convert these newly-described options into changes to the system configuration.
It may help if I showed you the concrete options that my newly-created blog module provides:

{{< codelines "Nix" "blog-static-flake/module.nix" 32 43 >}}

There are three options here:
* `enable`, a boolean-valued input that determines whether or not
the module should make any changes to the system configuration at all.
* `sites`, which, as written in the code, accepts a list of derivations.
  These derivations correspond to the various versions of my site that should
  be served to the outside world.
* `challengePath`, a string to configure where ACME will place files during
  automatic SSL renewal.

Now, while these are the only three options the user will need to set, the changes
to the system configuration are quite involved. For instance, for each site (derivation)
in the `sites` list, the resulting configuration needs to have a `virtualHost` in
the `services.nginx` namespace. To this end, I defined a function that accepts
a site derivation and produces the necessary settings:

{{< codelines "Nix" "blog-static-flake/module.nix" 7 19 >}}

Each virtual host always has a `root` option (where Nginx should look for HTML files),
but only those sites for which SSL is enabled need to specify `addSSL`, `enableACME`,
and `acmeRoot`. All the virtual hosts are assembled into a single array (below,
`cfg` refers to the options that the user provided to the module, as specified above).

{{< codelines "Nix" "blog-static-flake/module.nix" 28 28 >}}

If the `enable` option is set, we enable Nginx, and provide it with a list of all of the
virtual hosts we generated. Below, `config` (not to be confused with `cfg`) is the
namespace for the module's configuration.

{{< codelines "Nix" "blog-static-flake/module.nix" 45 51 >}}

In a similar manner to this, I generate a list of `systemd` services which are used
to configure the challenge path to be writeable. Click the `module.nix` link above
to check out the full file.

#### Creating a Flake
We now have two "things" that handle the deployment of the blog:
the builder functions `english` and `russian` which help describe various
blog versions, and the NixOS module that configures the server's Nginx to
serve said versions. We now want to expose these to the NixOS system configuration,
which describes the entire server. This is where flakes finally come in.
[Yanik Sander](https://blog.ysndr.de/posts/internals/2021-01-01-flake-ification/index.html) wrote up
a pretty comprehensive explanation of how their blog is deployed using flakes, which I often consulted
while getting started -- check it out if you are looking for more details.

In brief, a Nix flake has _inputs_ and _outputs_. Inputs can be other flakes or source files that
the flake needs access to, and outputs are simply Nix expressions that the flake provides.

The nice thing about flakes' inputs is that they can reference other flakes via Git. This means
that, should I write a flake for my blog (as I am about to do) I will be able to reference its git URL
in another flake, and Nix will automatically clone and import it. This helps achieve the __adding custom packages__
goal, since I can now easily write Nix expressions and reference them from my system configuration.

Importantly, flakes track the versions of their inputs in a `flake.lock` file; this means that, unless explicitly told to do otherwise,
they will use the same version of their inputs. This achieves the __versioning__ goal for my blog, too, since
now it will pull the pre-defined commit from Git until I tell it to fetch the updated site.
In addition to pinning the version of my blog, though, the flake also locks down the version of `nixpkgs` itself.
This means that the same packages will be used in the build process, instead of those found on the host
system at the time. This has the nice effect of preventing updates to dependencies from breaking the build;
it's a nice step towards purity and reproducibility.

Let's take a look at the inputs of my blog flake:

{{< codelines "Nix" "blog-static-flake/flake.nix" 2 19 >}}

Two of these inputs are my blog source code, pulled from its usual Git host. They are marked as
`flake = false` (my blog is just a Hugo project!), and both require submodules to be fetched. One
of them is set to the `localization` branch, once again because localization is not yet stabilized
and thus not merged into my blog's `master` branch. The other three inputs are flakes, one of
which is just `nixpkgs`. The `flake-utils` flake provides some convenient functions for writing other
flakes, and `katex-html` is my own creation, a KaTeX-to-HTML conversion script that I use to post-process the blog.

So what outputs should this flake provide? Well, we've already defined a NixOS module for the blog, and we'd like
our flake to expose this module to the world. But the module alone is not enough; its configuration requires a list of
packages created using our builders. Where does one procure such a list? The caller will need access to the builders
themselves. To make all of this work, I ended up with the following expression for my `outputs`:

{{< codelines "Nix" "blog-static-flake/flake.nix" 21 34 >}}

The flake `output` schema provides a standard option for exposing modules, `nixosModule`. Then,
exposing my `module.nix` file from the flake is simply a matter of importing it, as on line 31.
There is, however, no standard way for exposing a _function_. The good news is that any
attribute defined on a flake is accessible from code that imports that flake. Thus, I simply
added a `buildersFor` function, which fetches the `nixpkgs` collection
and LaTeX builder script for a given system, and feeds them to the file that defines the `english`
and `russian` builders. This `buildersFor` function also provides the builders with the two
different blog sources they reference, `blog-source` and `blog-source-localized`.

The `system` parameter to `buildersFor` is necessary because the set of packages from `nixpkgs` depends on it. Thus, if the
builders use any packages from the collection (they do), they must know which system to pull packages for.
This is a common pattern in flakes: the `packages` attribute is typically a system-to-package mapping, too.

Finally, the last little bit on lines 32 through 34 defines a default package for the flake. This
is the package that is built if a user runs `nix build .#`. This isn't strictly necessary for my purposes,
but it's nice to be able to test that the builders still work by running a test build. The
`eachDefaultSystem` function generates a `defaultPackage` attribute for each of the "default"
systems, so that the package is buildable on more than just my server architecture.

And that's it for the blog flake! I simply push it to Git, and move on to actually _using_ it from elsewhere.

#### Using the Module

In my server configuration (which is, itself, a flake), I simply list my `blog-static-flake` as one
of the inputs:

{{< codelines "Nix" "server-config/flake.nix" 4 4 >}}

Then, in the `modules` attribute, I include `blog.nixosModule`, making NixOS aware of its options and configuration.
The final little piece is to provide the `english` and `russian` builders to the system configuration;
this can be done using the `specialArgs` attribute. The whole `flake.nix` file is pretty short:

{{< codeblock "Nix" "server-config/flake.nix" >}}

Finally, in `configuration.nix`, taking `builders` as one of the inputs, I write what you saw above:

{{< codelines "Nix" "server-config/configuration.nix" 42 59 >}}

### Wrapping Up
So there you have it, a flake-based multi-version blog deployment written in a declarative style. You can
check out both my [system configuration flake](https://dev.danilafe.com/Nix-Configs/server-config)
and my [blog flake](https://dev.danilafe.com/Nix-Configs/blog-static-flake) on my Git server.
If you want more, check out the articles by Xe and Yannik linked above. Thanks for reading!