diff --git a/.gitmodules b/.gitmodules index 90f801d..090ff14 100644 --- a/.gitmodules +++ b/.gitmodules @@ -10,3 +10,6 @@ [submodule "code/server-config"] path = code/server-config url = https://dev.danilafe.com/Nix-Configs/server-config +[submodule "code/blog-static-flake"] + path = code/blog-static-flake + url = vanilla@dev.danilafe.com:Nix-Configs/blog-static-flake.git diff --git a/code/blog-static-flake b/code/blog-static-flake new file mode 160000 index 0000000..f2bb36b --- /dev/null +++ b/code/blog-static-flake @@ -0,0 +1 @@ +Subproject commit f2bb36b862b7c730c42abb0b759ddd3c1c4f3d4b diff --git a/content/blog/blog_with_nix.md b/content/blog/blog_with_nix.md index 2e3226d..8cd75ae 100644 --- a/content/blog/blog_with_nix.md +++ b/content/blog/blog_with_nix.md @@ -1,7 +1,6 @@ --- title: "Declaratively Deploying Multiple Blog Versions with NixOS and Flakes" date: 2021-10-23T18:01:31-07:00 -expirydate: 2021-10-23T18:01:31-07:00 draft: true tags: ["Hugo", "Nix"] --- @@ -15,7 +14,7 @@ or {{< sidenote "right" "pcr-note" "may or may not be COVID™." >}} The results of the PCR test are pending at the time of writing. {{< /sidenote >}} -In seeming correspondence with the progression of my cold, a thought occured in the back of my mind: +In seeming correspondence with the progression of my cold, a thought occurred in the back of my mind: "_Your blog deployment is kind of a mess_". On the first day, when I felt only a small tingling in my throat, I waved that thought away pretty easily. On the second day, feeling unwell and staying in bed, I couldn't help but start to look up Nix documentation. And finally, on the third day, @@ -34,7 +33,7 @@ version of my blog. It's rather empty, because translation is hard work, so it o so far as another "draft" website. My build process (a derivative of what I describe in [rendering mathematics on the back end]({{< relref "./backend_math_rendering.md" >}})) is also fairly unconventional. When I developed this site, the best -form of server-side mathematics rendering was handlded by KaTeX, and required some additional +form of server-side mathematics rendering was handled by KaTeX, and required some additional work to get rolling (specifically, I needed to write code to replace sections of LaTeX on a page with their HTML and MathML versions). There may be a better way now, but I haven't yet performed any kind of migration. @@ -50,10 +49,12 @@ me use flakes? Well, two things: * __Adding custom packages__. The Nix code for my blog provides a package / derivation for each version of my website, and I want to use these packages in my `configuration.nix` so that I can point various Nginx virtual hosts to each of them. This is typically done using - overlays, but I need a clean way to let my system configuration pull in my blog overlay (or blog - packages); flakes solve this issue my letting me specify a blog flake, and pull it in as one + overlays; however, how should my system configuration get my overlay Nix expression? I would + like to be able to separate my build-the-blog code from my describe-the-server code, and + so I need a clean way to let my system access the former from the latter. + flakes solve this issue my letting me specify a blog flake, and pull it in as one of the inputs. -* __Versioning__. My process for deploying new versions of the site prior to flakes boiled down to fetcing +* __Versioning__. My process for deploying new versions of the site prior to flakes boiled down to fethcing the latest commit from the `master` branch of my blog repository, and updating the `default.nix` file with that commit. This way, I could reliably fetch the version of my site that I want published. Flakes do the same thing: the `flake.lock` file @@ -73,6 +74,152 @@ it's very clear from the configuration what I want from my server: three virtual hosts, one with HTTPS, one with drafts, and one with drafts and _in Russian_. Second, there's plenty of code reuse. I'm using two builder functions, `english` and `russian`, but under the hood, the exact same code is being -used to run Hugo and all the necessay post-processing. Finally, all of this can be +used to run Hugo and all the necessary post-processing. Finally, all of this can be used pretty much immediately given my blog flake, which reduces the amount of glue code I have to write. + +### Getting There +#### A Derivation Builder +As I mentioned earlier, I need to generate multiple versions of my blog. All of these +use pretty much the same build process -- run Hugo on the Markdown files, then do some +post-processing (in particular, convert the LaTeX in the resulting pages into MathML +and nice-looking HTML). I didn't want to write this logic multiple times, so I settled +for a function that takes some settings, and returns a derivation: + +{{< codelines "Nix" "blog-static-flake/lib.nix" 6 21 >}} + +There are a few things here: + +* On line 7, the settings `src`, `ssl`, and `host` are inherited into the derivation. + The `src` setting provides a handle on the source code of the blog. I haven't had + much time to test and fine-tune the changes enabling multi-language support on the site, + so they reside on a separate branch. It's up to the caller to provide which version of the + source code should be used for building. The `host` and `ssl` settings are interesting + because __they don't actually matter for the derivation itself__ -- they just aren't used + in the builder. However, attributes given to a derivation are accessible from "outside", + and these settings will play a role later. +* Lines 10 through 14 deal with setting the base URL of the site. Hugo, my static site generator, + does not know how to interpret + the `--baseURL` option when a blog has multiple languages. What this means is that in the end, + it is impossible to configure the base URL used in links from the command line, + and I need to apply some manual changes to the configuration file. I need to be able to adjust + the base URL becasue each version of my website is hosted in a different place: the default (english) + website is hosted on `danilafe.com`, the version with drafts on `drafts.danilafe.com`, and so on. + However, the configuration file only knows one base URL per language, and so it _doesn't_ know + when or when not to use the `drafts.` prefix. The `urlSub` variable is used in the builder. +* On line 15, the `publicPath` variable is set; while single-language Hugo puts all the generated + HTML into the `public` folder, the multi-language configuration places them into `public/[language-code]`. + Thus, depending on the configuration, the builer needs to look in a different place for final output. + +This new `website` function is general enough to represent all my blog versions, but it's too low-level. +Do I really want to specify the `publicPath` each time I want to describe a version of the site? +What about `settings.replaceUrl`, or the source code? Just like I would in any garden variety language, +I defined two helper functions: + +{{< codelines "Nix" "blog-static-flake/lib.nix" 25 46 >}} + +Both of these simply make a call to the `website` function (and thus return derivations), but they +make some decisions for the caller, and provide a nicer interface by allowing attributes to be omitted. +Specifically, by default, a site version is assumed to be HTTP-only, and to contain non-draft articles. +Furthermore, since each function corresponds to a language, there's no need for the caller to provide +a blog version, and thus also the output `path`, or even to specify the "from" part of `replaceUrl`. +The `wrapHost` function, not included in the snippet, simply adds `http` or `https` to the `host` +parameter, which does not otherwise include this information. These functions can now be called +to describe different versions of my site: + +```Nix +# Default version, hosted on the main site and using HTTPS +english { + ssl = true; + host = "danilafe.com"; +} + +# English draft version, hosted on draft domain and not using HTTPS. +english { + drafts = true; + host = "drafts.danilafe.com"; +} + +# Russian draft version, hosted on draft (russian) domain, and not using HTTPS. +russian { + drafts = true; + host = "drafts.ru.danilafe.com"; +} +``` + +#### Configuring Nginx +The above functions are already a pretty big win (in my opinion) when it comes to +describing my blog. However, by themselves, they aren't quite enough to clean +up my system configuration: for each of these blog versions, I'd need to add +an Nginx `virtualHosts` entry where I'd pass in the corresponding host (like `danilafe.com` or +`drafts.danilafe.com`), configure SSL, and so on. At one point, too, all paths +in `/var` were by default mounted as read-only by NixOS, which meant that it was necessay +to tell `systemd` that `/var/www/challenges` should be writeable so that the SSL +certificate for the site could be properly renewed. Overall, this was a lot of detail +that I didn't want front-and-center in my server configuration. + +However, with the additional "ghost" attributes, my derivations already contain most +of the information required to configure Nginx. The virtual host, for instance, +is the same as `replaceUrl.to` (since I'd want the Nginx virtual host for a blog +version to handle links within that version). The `ssl` ghost parameter corresponds +precisely to whether or not a virtual host will need SSL (and thus ACME, and thus +the `systemd` setting). + +To make this _really_ nice, I wanted all of this to be "just another section of my +configuration file". That is, I wanted to control my site deployment via regular +old attributes in `configuration.nix`. To this end, I needed a module. Xe recently +[wrote about NixOS modules in flakes](https://christine.website/blog/nix-flakes-3-2022-04-07), +and what I do here is very similar. In essence, a module has two bits: + +* The _options_, which specify what kind of attributes this module understands. + The most common option is `enable`, which tells a module that it should apply + its configuration changes. +* The _configuration_, which consists of the various system settings that this module + will itself set. These typically depend on the options. + +In short, a module describes the sort of options it will accept, and then provides +a way to convert these newly-described options into changes to the system configuration. +It may help if I showed you the concrete options that my newly-created blog module provides: + +{{< codelines "Nix" "blog-static-flake/module.nix" 32 43 >}} + +There are three options here: +* `enable`, a boolean-valued input that determines whether or not +the module should make any changes to the system configuration at all. +* `sites`, which, as written in the code, accepts a list of derivations. + These derivations correspond to the various versions of my site that should + be served to the outside world. +* `challengePath`, a string to configure where ACME will place files during + automatic SSL renewal. + +Now, while these are the only three options the user will need to set, the changes +to the system configuration are quite involved. For instance, for each site (derivation) +in the `sites` list, the resulting configuration needs to have a `virtualHost` in +the `services.nginx` namespace. To this end, I defined a function that accepts +a site derivation and produces the necessary settings: + +{{< codelines "Nix" "blog-static-flake/module.nix" 7 19 >}} + +Each virtual host always has a `root` option (where Nginx should look for HTML files), +but only those sites for which SSL is enabled need to specify `addSSL`, `enableACME`, +and `acmeRoot`. All the virtual hosts are assembled into a single array (below, +`cfg` refers to the options that the user provided to the module, as specified above). + +{{< codelines "Nix" "blog-static-flake/module.nix" 28 28 >}} + +If the `enable` option is set, we enable Nginx, and provide it with a list of all of the +virtual hosts we generated. Below, `config` (not to be confused with `cfg`) is the +namespace for the module's configuration. + +{{< codelines "Nix" "blog-static-flake/module.nix" 45 51 >}} + +In a similar manner to this, I generate a list of `systemd` services which are used +to configure the challenge path to be writeable. Click the `module.nix` link above +to check out the full file. + +#### Creating a Flake +{{< todo >}} This needs to be done {{< /todo >}} + +#### Using the Module + +{{< codelines "Nix" "server-config/configuration.nix" 42 59 >}}