9 minutes
Bootstrapping guix on Kimsufi
UPDATE This post is outdated now, follow this https://guix.gnu.org/en/cookbook/en/html_node/Running-Guix-on-a-Kimsufi-Server.html
This is a short guide on how to get guix up and running on a Kimsufi server.
If only Kimsufi had a Bring Your Own Image feature I would not be writing this post, it took me an entire afternoon to try the usual install the system in rescue mode and that did not work.
Jump straight to Converting Debian into Guix if you just want guix up and running.
Rescue method
The manual way
From the Web UI you’re supposed to change netboot to rescue64-pro then restart the machine, OVH will then mail you with the credentials needed to ssh into the server.
Once that is done you have access to a minimal debian with qemu installed and from there you can partition the disks, download the guix iso, launch it with qemu and continue the installation through vnc.
See this post for a step-by-step guide.
I chose to write a shell script that would automate the bootstrap process.
It’s not just a plug and play script you will have to adapt it to your own config (change the ip addresses, the services that you use…) and transfer it plus the os.scm file via sftp to the server.
#!/usr/bin/env bash
# See https://guix.gnu.org/cookbook/en/html_node/Running-Guix-on-a-Linode-Server.html
# From the webui -> netboot rescue rescue64-pro -> restart -> get the credentials in the mail
# -> ssh into the machine
# Guix
wget https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
chmod +x guix-install.sh
./guix-install.sh
guix pull
# Partitions
# See https://blog.hqcodeshop.fi/archives/273-GNU-Parted-Solving-the-dreaded-The-resulting-partition-is-not-properly-aligned-for-best-performance.html
# https://wiki.archlinux.org/title/Advanced_Format#Partition_alignment
mdadm --stop /dev/md127
mdadm --zero-superblock /dev/sda2 /dev/sdb2
wipefs -a /dev/sda
wipefs -a /dev/sdb
parted /dev/sda --align=opt -s -m -- mklabel gpt
parted /dev/sda --align=opt -s -m -- \
mkpart bios_grub 1049kb 512MiB \
set 1 bios_grub on
parted /dev/sda --align=opt -s -m -- \
mkpart primary 512MiB -512MiB
set 2 raid on
parted /dev/sda --align=opt -s -m -- mkpart primary linux-swap 512MiB 100% # Swap
parted /dev/sdb --align=opt -s -m -- mklabel gpt
parted /dev/sdb --align=opt -s -m -- \
mkpart bios_grub 1049kb 512MiB \
set 1 bios_grub on
parted /dev/sdb --align=opt -s -m -- \
mkpart primary 512MiB -512MiB \
set 2 raid on
parted /dev/sdb --align=opt -s -m -- mkpart primary linux-swap 512MiB 100% # Swap
## RAID
## Use btrfs instead of mdadm?
mdadm --create /dev/md127 --level=1 --raid-disks=2 --metadata=0.90 /dev/sda2 /dev/sdb2
## Boot partitions
mkfs.ext4 /dev/sda1
mkfs.ext4 /dev/sdb1
## Root filesystem
mkfs.ext4 /dev/md127
## Swap partitions
mkswap /dev/sda3
swapon /dev/sda3
mkswap /dev/sdb3
swapon /dev/sdb3
# Installing guix system
mkdir /mnt/guix
mount /dev/md127 /mnt/guix
guix system init /root/os.scm /mnt/guix
# Need to wait 5-10 mn to reboot
Here is an example of an os definition:
(use-modules (gnu))
(use-service-modules networking ssh vpn virtualization sysctl admin)
(use-package-modules ssh certs tls tmux vpn virtualization)
(operating-system
(host-name "kimsufi")
(timezone "Etc/UTC")
(bootloader (bootloader-configuration
(bootloader grub-bootloader)
;(targets (list "/dev/sda" "/dev/sdb"))
(targets (list "/dev/sda"))
(terminal-outputs '(console))))
;; Add a kernel module for RAID-1 (aka. "mirror").
(initrd-modules (cons* "raid1" %base-initrd-modules))
;; (mapped-devices
;; (list
;; (mapped-device
;; ; (source (list "/dev/sda2" "/dev/sdb2"))
;; (target "/dev/md2")
;; (type raid-device-mapping))))
(swap-devices
(list
(swap-space
(target "/dev/sda3"))
;; (swap-space
;; (target "/dev/sdb3"))
))
(issue
;; Default contents for /etc/issue.
"
This is the GNU system at Kimsufi. Welcome.\n")
(file-systems (cons* (file-system
(mount-point "/")
(device (file-system-label "root"))
;(device "/dev/md2")
;(device "/dev/sda2")
(type "ext4")
;(dependencies mapped-devices)
)
%base-file-systems))
(users (cons (user-account
(name "guix")
(comment "guix")
(group "users")
(supplementary-groups '("wheel"))
(home-directory "/home/guix"))
%base-user-accounts))
(sudoers-file
(plain-file "sudoers" "\
root ALL=(ALL) ALL
%wheel ALL=(ALL) ALL
guix ALL=(ALL) NOPASSWD:ALL\n"))
;; Globally-installed packages.
(packages (cons* tmux nss-certs gnutls wireguard-tools %base-packages))
(services
(cons*
(service static-networking-service-type
(list (static-networking
(addresses (list (network-address
(device "enp3s0")
(value "REPLACE_ME"))
(network-address
(device "enp3s0")
(value "REPLACE_ME"))))
(routes (list (network-route
(destination "default")
(gateway "37.187.79.254"))
(network-route
(destination "default")
(gateway "2001:41d0:a:2fFF:FF:FF:FF:FF"))))
(name-servers '("213.186.33.99")))))
(service unattended-upgrade-service-type)
(service openssh-service-type
(openssh-configuration
(openssh openssh-sans-x)
(permit-root-login #f)
(port-number 22222)
(authorized-keys
(quasiquote
(("REPLACE_ME" (unquote (plain-file "REPLACE_ME.pub" "SSH_KEY.PUB"))))))))
(modify-services %base-services
(sysctl-service-type config =>
(sysctl-configuration
(settings (append '(("net.ipv6.conf.all.autoconf" . "0")
("net.ipv6.conf.all.accept_ra" . "0"))
%default-sysctl-settings))))))))
Now everything should be good to go, the only thing left is to change the netboot back to hard drive and then reboot the machine.
And……………. after rebooting it turns out that I can ping the machine but all ports are closed.
I did reboot the machine in rescue mode again, mount the guix system into /mnt
hoping to see some logs, but there was not anything, the logs files were simply
not there, neither was the /home/user/guix directory.
So I guess it simply cannot boot but as for why? I have no idea.
Since kimsufi does not provide you with a KVM console you’re kinda screwed.
The automated way
Now I did not want to do it the manual way, I wanted something that could be automated with pulumi and some ansible/guile/shell script (ideally everything would be done via guix and guile but there is sadly too much to reimplement at the moment).
Retrieve the SSH credentials
I examined the kimsufi api and there are 3 endpoints of interest:
- /me/notification/email/history/
- /me/notification/email/history/{id}/
- /secret/
The first endpoint allow you to retrieve alls the ids of the emails that ovh sent you, from there you can feed the last id to the second endpoint.
This will returns a body of text with the email containing the credentials, you will have to parse it in order to retrive the secret uuid.
Once this is done you can query the secret endpoint with the uuid and that will return the ssh password in plain text.
Pulumi
Well now it’s time to write a program in one of the language supported by pulumi I decided to use python since I have to write fewer lines than say Java/C#.
I might have took Go if this was something more important but for my little infra I don’t care that much.
First step it to get the ovh provider for pulumi, this provider will allow us to query the aforementionned endpoints. Also install the command provider to run an ansible playbook that will take care of partitioning/running various guix commands…
Now the ovh provider is a terraform only thing, the pulumi provider for ovh is actually entirely generated from that terraform provider.
I could unfortunately not get it to work, when I would run pulumi up it would
crash with a DistroNotFoundError.
So I just gave up because I had other problems in the manual way and I did not want to use terraform again.
Converting Debian into Guix
From the Web UI you’ll need to install either a Debian 10 or Debian 11 machine.
Once that is done you can adapt the script, transfer via sftp and run it.
Now there are gotchas when bootstraping from a live os like this.
If you try to bootstrap with a full configuration with dozen of services you will get weird errors, so it’s better to start with something minimal and once the system is up reconfigure with the full configuration you want.
That is why libvirt, virtlog… are commented out, if I did not I would otherwise get a weird dbus error.
#!/usr/bin/env bash
# Target a debian10 machine
sudo su -
apt-get update
apt-get install xz-utils -y
wget https://ftp.gnu.org/gnu/guix/guix-binary-1.4.0.x86_64-linux.tar.xz
cd /tmp
tar --warning=no-timestamp -xvf ~/guix-binary-1.4.0.x86_64-linux.tar.xz
mv var/guix /var/ && mv gnu /
mkdir -p /root/.config/guix
ln -sf /var/guix/profiles/per-user/root/current-guix ~root/.config/guix/current
export GUIX_PROFILE="/root/.config/guix/current" ;
source $GUIX_PROFILE/etc/profile
groupadd --system guixbuild
for i in `seq -w 1 10`; do
useradd -g guixbuild -G guixbuild \
-d /var/empty -s `which nologin` \
-c "Guix build user $i" --system \
guixbuilder$i;
done;
cp -v /root/.config/guix/current/lib/systemd/system/guix-daemon.service /etc/systemd/system/
systemctl start guix-daemon && systemctl enable guix-daemon
mkdir -p /usr/local/bin
cd /usr/local/bin
ln -s /var/guix/profiles/per-user/root/current-guix/bin/guix
mkdir -p /usr/local/share/info
cd /usr/local/share/info
for i in /var/guix/profiles/per-user/root/current-guix/share/info/*; do
ln -s $i;
done
guix archive --authorize < /root/.config/guix/current/share/guix/ci.guix.gnu.org.pub
guix pull
guix install glibc-utf8-locales-2.29 openssl glibc-locales
export GUIX_LOCPATH="$HOME/.guix-profile/lib/locale"
# Yeah I actually have to comment out libvirt and virtlog for now
# otherwise I get a werid libvirt error.
# Reenable them once we've succesfully bootstraped.
cat > /etc/bootstrap-config.scm << EOF
(use-modules (gnu))
(use-service-modules networking ssh vpn virtualization sysctl certbot admin)
(use-package-modules ssh certs tls tmux vpn virtualization)
(operating-system
(host-name "guix")
(timezone "Etc/UTC")
(bootloader (bootloader-configuration
(bootloader grub-bootloader)
(targets (list "/dev/sda" "/dev/sdb"))
(terminal-outputs '(console))))
;; Add a kernel module for RAID-1 (aka. "mirror").
(initrd-modules (cons "raid1" %base-initrd-modules))
(mapped-devices
(list
(mapped-device
(source (list "/dev/sda2" "/dev/sdb2"))
(target "/dev/md2")
(type raid-device-mapping))))
(swap-devices
(list
(swap-space
(target "/dev/sda3"))
(swap-space
(target "/dev/sdb3"))))
(issue
;; Default contents for /etc/issue.
"
This is the GNU system at Kimsufi. Welcome.\n")
(file-systems (cons* (file-system
(mount-point "/")
(device "/dev/md2")
(type "ext4")
(dependencies mapped-devices))
%base-file-systems))
(users (cons (user-account
(name "debian")
(comment "debian")
(group "users")
;(supplementary-groups '("wheel"))
;(supplementary-groups '("wheel" "libvirt" "kvm"))
(home-directory "/home/debian"))
%base-user-accounts))
(sudoers-file
(plain-file "sudoers" "\
root ALL=(ALL) ALL
%wheel ALL=(ALL) ALL
debian ALL=(ALL) NOPASSWD:ALL\n"))
;; Globally-installed packages.
(packages (cons* tmux nss-certs gnutls wireguard-tools %base-packages))
(services
(cons*
(service static-networking-service-type
(list (static-networking
(addresses (list (network-address
(device "enp3s0")
(value "37.187.79.64/24"))
(network-address
(device "enp3s0")
(value "2001:41d0:a:2f40::1/64"))))
(routes (list (network-route
(destination "default")
(gateway "37.187.79.254"))
(network-route
(destination "default")
(gateway "2001:41d0:a:2fFF:FF:FF:FF:FF"))))
(name-servers '("213.186.33.99")))))
; (service unattended-upgrade-service-type)
;
; (service nftables-service-type
; (nftables-configuration
; (ruleset
; (plain-file "nftables.nft"
; "\
;table ip nat {
; chain prerouting {
; type nat hook prerouting priority -100;
; tcp dport { http, https } dnat to 192.168.1.10:http
; }
;
; chain postrouting {
; type nat hook postrouting priority 100;
; masquerade
; }
;}
;
;table inet filter {
; chain input {
; type filter hook input priority 0; policy drop;
;
; # early drop of invalid connections
; ct state invalid drop
;
; # allow established/related connections
; ct state { established, related } accept
;
; # allow icmp
; ip protocol icmp accept
; ip6 nexthdr icmpv6 accept
;
; # allow from loopback
; iifname lo accept
;
; # added: make NAT from libvirt work
; iifname virbr0 accept
;
; # allow ssh,http
; tcp dport {http,https,53,67,2222} accept
; udp dport {53,67} accept
;
; # reject everything else
; reject with icmpx type port-unreachable
; }
; chain forward {
; type filter hook forward priority 0; policy drop;
; iifname virbr0 oifname enp3s0 accept
; iifname enp3s0 oifname virbr0 accept
; }
; chain output {
; type filter hook output priority 0; policy accept;
; }
;}"))))
;
; (service libvirt-service-type
; (libvirt-configuration
; (unix-sock-group "libvirt")
; (tls-port "16555")))
;
; (service virtlog-service-type
; (virtlog-configuration
; (max-clients 1000)))
(service openssh-service-type
(openssh-configuration
(port-number 2222)
(permit-root-login #f)))
(modify-services %base-services
(sysctl-service-type config =>
(sysctl-configuration
(settings (append '(("net.ipv6.conf.all.autoconf" . "0")
("net.ipv6.conf.all.accept_ra" . "0"))
%default-sysctl-settings))))))))
EOF
guix system build /etc/bootstrap-config.scm
# TODO: dbus
mv /etc/{ssl,pam.d,skel,udev} /tmp
guix system reconfigure /etc/bootstrap-config.scm
mv /etc /old-etc
mkdir /etc
cp -r /old-etc/{passwd,group,shadow,gshadow,mtab,guix,bootstrap-config.scm} /etc/
guix system reconfigure /etc/bootstrap-config.scm
# The users uid created by guix is set to 100 and the one made
# made by debian is 1000, so we change that for guix.
chown -R debian:users /home/debian
reboot
Now if everything has gone smoothly you should have a guix system up and running!