diff options
Diffstat (limited to 'content')
-rw-r--r-- | content/_index.md | 4 | ||||
-rw-r--r-- | content/pages/_index.md | 3 | ||||
-rw-r--r-- | content/pages/about.md | 54 | ||||
-rw-r--r-- | content/pages/projects.md | 9 | ||||
-rw-r--r-- | content/posts/2023-01-23-hello-world.md | 20 | ||||
-rw-r--r-- | content/posts/2023-01-31-nixos-zfs-mirrored-boot.md | 361 | ||||
-rw-r--r-- | content/posts/_index.md | 5 |
7 files changed, 456 insertions, 0 deletions
diff --git a/content/_index.md b/content/_index.md new file mode 100644 index 0000000..87f0dca --- /dev/null +++ b/content/_index.md @@ -0,0 +1,4 @@ ++++ +title = "Home" +sort_by = "date" ++++ diff --git a/content/pages/_index.md b/content/pages/_index.md new file mode 100644 index 0000000..800a244 --- /dev/null +++ b/content/pages/_index.md @@ -0,0 +1,3 @@ ++++ +render = false ++++ diff --git a/content/pages/about.md b/content/pages/about.md new file mode 100644 index 0000000..863ff92 --- /dev/null +++ b/content/pages/about.md @@ -0,0 +1,54 @@ ++++ +title = "About" +path = "about" + +[extra] +raw = true ++++ +## About me +Hi, I'm **sefidel** (sef). + +- PGP: <a href="/.well-known/openpgpkey/hu/dj3498u4hyyarh35rkjfnghbjxug6b19" download="sefidel.pgp"> +8BDF DFB5 6842 2393 82A0 441B 9238 BC70 9E05 516A + </a> +- Email: [contact@sefidel.net][email] [sef@exotic.sh][email-secondary] +- Fediverse: [@sefidel@stella.place][fedi-stella] (ko-KR) +- Matrix: [@sef:exotic.sh][matrix] [@sefidel:nixos.dev][matrix-secondary] (for + FOSS use only) +- Timezone: UTC+0900 +- Languages: English (en-GB), Korean (ko-KR), Japanese (ja-JP) + +***REMOVED*** + +My main interests are low-level systems, compilers, distributed computing, microservices and security. + +Feel free to contact me on Matrix (preferred) or email! +I'm also on IRC as `sefidel` (Libera, OFTC). + +Most of my works can be found on [exotic.sh git][git-exotic], +[GitHub][git-github], and on [SourceHut][git-srht]. + +## System +I mainly use MacBook Air M1 (2020) for development, as my NixOS workstation's CPU cooler is currently defunct. + +I use Nix for my system configuration, and my nixrc can be found [here][nixrc]. +You should be able to find configurations for most of the software I use there. + +My favourite choice of text editor is Neovim, and the configuration for it can +be found [here][nvimrc]. It's not managed with Nix, since Nix doesn't have +decent Lua config support (yet). +Plus, I sometimes have to use this configuration on non-nix systems. +If you're going to use this configuration, keep in mind that it looks best with +a bitmap font like [Dina]. + +[email]: mailto:contact@sefidel.net +[email-secondary]: mailto:sef@exotic.sh +[matrix]: https://matrix.to/#/@sef:exotic.sh +[matrix-secondary]: https://matrix.to/#/@sefidel:nixos.dev +[fedi-stella]: https://stella.place/@sefidel +[git-exotic]: https://git.exotic.sh/pub/sefidel +[git-github]: https://github.com/sefidel +[git-srht]: https://sr.ht/~sefidel +[nixrc]: https://git.exotic.sh/pub/sefidel/nixrc +[nvimrc]: https://git.exotic.sh/pub/sefidel/nvimrc +[Dina]: https://www.dcmembers.com/jibsen/download/61 diff --git a/content/pages/projects.md b/content/pages/projects.md new file mode 100644 index 0000000..98d370f --- /dev/null +++ b/content/pages/projects.md @@ -0,0 +1,9 @@ ++++ +title = "Projects" +path = "projects" + +[extra] +raw = true ++++ + +// TODO: Open source projects goes here diff --git a/content/posts/2023-01-23-hello-world.md b/content/posts/2023-01-23-hello-world.md new file mode 100644 index 0000000..8024895 --- /dev/null +++ b/content/posts/2023-01-23-hello-world.md @@ -0,0 +1,20 @@ ++++ +title = "Hello, world!" +date = "2023-01-23" + +[taxonomies] +categories = ["meta"] ++++ + +**Hello, world!** + +This is my first post on my personal website. + +Here's a quick overview of what will be posted here: + +- Status Updates +- Development notes +- Generally anything I find interesting (they'll be tagged accordingly) + +I have a broad interest in technology, so if you have something to share, please +don't hesitate to [reach out](@/pages/about.md) to me! diff --git a/content/posts/2023-01-31-nixos-zfs-mirrored-boot.md b/content/posts/2023-01-31-nixos-zfs-mirrored-boot.md new file mode 100644 index 0000000..99b4004 --- /dev/null +++ b/content/posts/2023-01-31-nixos-zfs-mirrored-boot.md @@ -0,0 +1,361 @@ ++++ +title = "Installing NixOS with ZFS mirrored boot" +date = "2023-01-31" + +[taxonomies] +categories = ["system"] +tags = ["linux", "nixos"] ++++ + +// TODO: add PlantUML diagrams + +## Overview + +In this post, we're going to set up a ZFS mirrored boot system with full-disk encryption that is unlockable remotely. + +## Preparing the installation medium + +This step may vary depending on what system you're going to install NixOS into. + +This post assumes that you're installing this on a normal server, with a +minimal NixOS image. + +The community-maintained [NixOS wiki][nixos-wiki] contains guides to install +NixOS to devices in other conditions, such as a server with only remote access. + +You will need a USB stick before proceeding to the next step. + +First, download the latest NixOS image, and flash it: + +```sh +$ curl -L https:#channels.nixos.org/nixos-unstable/latest-nixos-minimal-x86_64-linux.iso -O nixos.iso +$ dd if=./nixos.iso of=/dev/sdX bs=1M status=progress +``` + +If your target machine architecture is not `x86_64`, replace it with your +desired architecture (e.g. `i686`, `aarch64`). + +After the image has been successfully flashed into your installation medium, +unplug it and boot using the medium on the target machine. + +## Preparing Disks + +We'll start by defining variables pointing to each disk by ID. + +According to the [Archlinux.org Wiki][arch-wiki], If you create zpools using device names +(e.g. `/dev/sda`), ZFS might not be able to detect zpools intermittently on +boot. + +You can grab the ID via `ls -lh /dev/disk/by-id/`. + +```sh +DISK1=/dev/disk/by-id/ata-VENDOR-ID-OF-THE-FIRST-DRIVE +DISK2=/dev/disk/by-id/ata-VENDOR-ID-OF-THE-SECOND-DRIVE +``` + +### Partitioning + +Then we'll partition our disks. Since this is a mirrored setup, we'll have to do +the exactly same operation twice. Fortunately, bash function come into rescue. + +The partition structure is the following: +``` +1GiB Boot | ~Remaining ZFS +``` + + +```sh +partition() { + sgdisk --zap-all "$1" + sgdisk -n 1:0:+1GiB -t 1:EF00 -c 1:boot "$1" + # Swap is omitted. + sgdisk -n 2:0:0 -t 2:BF01 -c 2:zfs "$1" + sgdisk --print "$1" +} + +partition $DISK1 +partition $DISK2 +``` + +### Creating vfat filesystem for boot + +Boot partitions should be formatted with 'vfat', in order for it to mount and +function without issues. + +```sh +mkfs.vfat $DISK1-part1 +mkfs.vfat $DISK2-part1 +``` + +### Configuring ZFS pool + +This dataset structure is based on [Erase your darlings][erase-your-darlings]. + +Now that we're done partitioning our disks, we'll create a ZFS pool named +'rpool', which is mirrored. This will prompt you to enter a passphrase for your +new ZFS pool. +```sh +zpool create \ + -o ashift=12 \ + -O mountpoint=none -O atime=off -O acltype=posixacl -O xattr=sa \ + -O compression=lz4 -O encryption=aes-256-gcm -O keyformat=passphrase \ + rpool mirror \ + $DISK1-part2 $DISK2-part2 +``` + +Then, we create a 'root dataset' which is `/ (root)` for the target machine, +then snapshot the empty state as 'blank'. +```sh +zfs create -p -o mountpoint=legacy rpool/local/root +zfs snapshot rpool/local/root@blank +``` + +Note the 'local' after rpool. In this setup, 'local' is treated as unimportant +data, i.e. packages, root, etc., Whereas 'safe' is treated as important data, +which needs to be backed up. + +And mount it: +```sh +mount -t zfs rpool/local/root /mnt +``` + +Then we mount the multiple boot partitions we created: +```sh +mkdir /mnt/boot +mkdir /mnt/boot-fallback + +mount $DISK1-part1 /mnt/boot +mount $DISK2-part1 /mnt/boot-fallback +``` + +Create and mount a dataset for `/nix`: +```sh +zfs create -p -o mountpoint=legacy rpool/local/nix +mkdir /mnt/nix +mount -t zfs rpool/local/nix /mnt/nix +``` + +And a dataset for `/home`: +```sh +zfs create -p -o mountpoint=legacy rpool/safe/home +mkdir /mnt/home +mount -t zfs rpool/safe/home /mnt/home +``` + +And a dataset for states that needs to be persisted between boots: +```sh +zfs create -p -o mountpoint=legacy rpool/safe/persist +mkdir /mnt/persist +mount -t zfs rpool/safe/persist /mnt/persist +``` + +Note: All states will be wiped each boot after setting up +[these](#erasing-your-darlings). +Make sure to put states that need to persist on `/persist`. + + +## Configuring NixOS + +Now that we're done with partitions and ZFS, it's time to declaratively +configure the machine. This step may vary depending on your machine, +please consult the docs when in doubt. + +### Getting the base configuration + +In this post, we're going to use plain `nixos-generate-config` to get a base +configuration files for the machine. + +```sh +nixos-generate-config --root /mnt +``` + +### Erasing your darlings + +In the [previous step](#configuring-zfs-pool), we've made a snapshot of blank +root to roll back to it each boot, to keep the system stateless. + +Add this to the `configuration.nix` to wipe the root dataset on each boot by +rolling back to the blank snapshot after the devices are made available: +```nix +{ + boot.initrd.postDeviceCommands = lib.mkAfter '' + zfs rollback -r rpool/local/root@blank + ''; +} +``` + +### Configuring Bootloader + +In order to get ZFS to work, we need the following options to be set: +```nix +{ + boot.supportedFilesystems = [ "zfs" ]; + networking.hostId = "<8 random chars>"; +} +``` + +You can grab your machine ID at `/etc/machine-id` for the `hostId`. + +Then we'll configure grub: +```nix +{ + # Whether installer can modify the EFI variables. + # If you encounter errors, set this to `false`. + boot.loader.efi.canTouchEfiVariables = true; + + boot.loader.grub.enable = true; + boot.loader.grub.efiSupport = true; + boot.loader.grub.device = "nodev"; + + # This should be done automatically, but explicitly declare it just in case. + boot.loader.grub.copyKernels = true; + # Make sure that you've listed all of the boot partitions here. + boot.loader.grub.mirroredBoots = [ + { path = "/boot"; devices = ["/dev/disk/by-uuid/<ID-HERE>"]; } + { path = "/boot-fallback"; devices = ["/dev/disk/by-uuid/<ID-HERE>"]; } + # ... + ]; +} +``` + +### Handling boot partitions gracefully + +By default, NixOS will throw an error and complain about it when there is a +missing partition/disk. Since we want the server to boot smoothly even if there +is a missing boot partition, so we need to set the 'nofail' option to those +partitions: + +```nix +{ + fileSystems."/boot".options = [ "nofail" ]; + fileSystems."/boot-fallback".options = [ "nofail" ]; +} +``` + + +### Enabling Remote ZFS Unlock + +On each boot, ZFS will ask for a passphrase to unlock the ZFS pool. +To work around this issue, we can start an SSH server in `initrd`, that is going +to live until the pool is unlocked. + +Note: If you rename the keys after, you may have some trouble rolling back to +previous generations: See [here](caveat-remote-unlock) for details. + +To achieve that, we'll first have to generate an SSH host key for the initrd: +```sh +ssh-keygen -t ed25519 -N "" -f /mnt/boot/initrd-ssh-key + +# Each boot partition should have the same key +cp /mnt/boot/initrd-ssh-key /mnt/boot-fallback/initrd-ssh-key +``` + +Then configure `initrd`: +```nix +{ + boot.kernelModules = [ "<YOUR-NETWORK-CARD>" ]; + boot.initrd.kernelModules = [ "<YOUR-NETWORK-CARD>" ]; + + # DHCP Configuration, comment on Static IP + networking.networkmanager.enable = false; + networking.useDHCP = true; + + # Uncomment on Static IP + # boot.kernelParams = [ + # # See <https:#www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt> for documentation. + # # ip=<client-ip>:<server-ip>:<gw-ip>:<netmask>:<hostname>:<device>:<autoconf>:<dns0-ip>:<dns1-ip>:<ntp0-ip> + # # The server ip refers to the NFS server -- not needed in this case. + # "ip=<YOUR-IPV4-ADDR>::<YOUR-IPV4-GATEWAY>:<YOUR-IPV4-NETMASK>:<YOUR-HOSTNAME>-initrd:<YOUR-NETWORK-INTERFACE>:off:<DNS-IP>" + # ]; + + boot.initrd.network.enable = true; + boot.initrd.network.ssh = { + enable = true; + + # Using the same port as the actual SSH will cause clients to throw errors + # related to host key mismatch. + port = 2222; + + # This takes 'path's, not 'string's. + hostKeys = [ + /boot/initrd-ssh-key + /boot-fallback/initrd-ssh-key + # ... + ]; + + # Public ssh key to log into the initrd ssh + authorizedKeys = [ "<YOUR-SSH-PUBKEY>" ]; + }; + boot.initrd.network.postCommands = '' + cat <<EOF > /root/.profile + if pgrep -x "zfs" > /dev/null + then + zfs load-key -a + killall zfs + else + echo "ZFS is not running -- this could be a sign of failure." + fi + EOF + ''; +} +``` + +## Installing NixOS + +Run `nixos-install`, then reboot your machine. + +Note: Make sure that you've configured SSH and network for your machine, +failure to do so may result in an inaccessible system. + +That's it! Enjoy your fresh NixOS machine! + +## Troubleshooting + +### Failed to import pool - more than one matching pool + +This error might occur when + +- one of your disks were previously used in another ZFS pool, and its metadata +weren't properly removed +- you messed up during install, and you repartitioning the disk without removing + its ZFS metadata. + +This is because the ZFS metadata doesn't live on a partition, but on a disk. + +Note: the following operations will irrevocably delete ANY data on your disk! + +To remove those left behind: + +```sh +sgdisk --zap-all $DISK +# Overwrite first 256M of the disk, removing metadata +# In some cases just `wipefs -a` works, but I found this to be the most +# reliable way to wipe them no matter what operations were performed on the disk +# before. +dd if=/dev/urandom bs=1M count=256 of=$DISK +``` + +And then you can try the installation again. + +## Conclusion + +## Acknowledgements + +I wrote this article because I've noticed that I always forget some steps +during NixOS installation to a newly acquired server. + +I've compiled resources listed below to make a step-by-step guide for a setup I +find 'optimal'. Please do check out those resources! + +- [NixOS Discourse Thread][discourse-thread] +- [Erase your darlings][erase-your-darlings] +- [Remote, encrypted ZFS storage server with NixOS][hetzner-zfs] +- [Encrypted ZFS mirror with mirrored boot on NixOS][nixos-zfs-mirrored-boot] + +[erase-your-darlings]: https://grahamc.com/blog/erase-your-darlings +[nixos-wiki]: https://nixos.wiki +[arch-wiki]: https://wiki.archlinux.org +[caveat-remote-unlock]: https://github.com/NixOS/nixpkgs/issues/101462#issuecomment-1172926129 +[discourse-thread]: https://discourse.nixos.org/t/nixos-on-mirrored-ssd-boot-swap-native-encrypted-zfs/9215 +[hetzner-zfs]: https://mazzo.li/posts/hetzner-zfs.html +[nixos-zfs-mirrored-boot]: https://elis.nu/blog/2019/08/encrypted-zfs-mirror-with-mirrored-boot-on-nixos diff --git a/content/posts/_index.md b/content/posts/_index.md new file mode 100644 index 0000000..60885e1 --- /dev/null +++ b/content/posts/_index.md @@ -0,0 +1,5 @@ ++++ +title = "Posts" +sort_by = "date" +template = "posts.html" ++++ |