Skip to content

Draft: Mount the nix store from hut in compute nodes

Rodrigo Arias Mallo requested to merge shared-nix-store into master

This MR solves one of the remaining pieces to have a working cluster with SLURM and Nix.

It mounts the nix store from hut into the compute nodes owl1 and owl2 by using an overlay. The overlay exposes first the store on disk (needed for boot) and then looks for the files in the hut store using a read-only NFS mount. Example:

hut% nix eval nixpkgs#cowsay.outPath
"/nix/store/k3xhwh54bf9z6xxsdz32lhq4h7c5fimj-cowsay-3.7.0"
hut% ls /nix/store/k3xhwh54bf9z6xxsdz32lhq4h7c5fimj-cowsay-3.7.0
ls: cannot access '/nix/store/k3xhwh54bf9z6xxsdz32lhq4h7c5fimj-cowsay-3.7.0': No such file or directory
hut% ssh owl2 ls /nix/store | grep k3xhwh54bf9z6xxsdz32lhq4h7c5fimj-cowsay-3.7.0
hut% nix build nixpkgs#cowsay
hut% ssh owl2 ls /nix/store | grep k3xhwh54bf9z6xxsdz32lhq4h7c5fimj-cowsay-3.7.0
k3xhwh54bf9z6xxsdz32lhq4h7c5fimj-cowsay-3.7.0

This allows users to run SLURM jobs in the compute nodes reading the dependencies directly from the hut store via NFS, avoiding the need to copy the clousure for each execution.

It is also posible to run nix develop directly from the compute nodes, but it is not recommended as it will replicate the same data in every compute node local disk.

Closes #23

Edited by Rodrigo Arias Mallo

Merge request reports