# Playing with Nix in adverse HPC environments

Presentation for FOSDEM 2023.

What happens when you have access to large clusters, but have little control
over the software installed on the machines? Unfortunately, this is the current
scenario that researchers often find in HPC clusters, which include very old
software stack, a brittle environment and non-cooperative sysadmins. We have
been experimenting with Nix to provide an up-to-date system running on top of
the already existing software, without root permissions with the help of user
namespaces.

In this talk we give a tour on the problems we found and how we solved them.
- Starting from the installation and configuration of nix to be used by multiple
  users when we lack a shared /nix store.
- Avoiding library contamination from /usr/lib with an isolated root mount
- Interactive development while compiling the code inside the isolated
  environment with a patched nix-portable
- Adding custom compilers to the stdenv
- Building packages tuned to an specific CPU with vectorization optimizations in
  mind.
- Running the benchmarks with SLURM inside the isolated environment with
  multiple compute nodes.
- Improving MPI fast zero-copy transfer inside user namespaces