Using Nix and nixpkgs-mozilla for Rust projects
TL;DR: Go to the Implement a shell.nix with nixpkgs-mozilla section
Introduction – why use nix if rustup exists?
Rust dependency management is top class, rustup
facilitates downloading and managing different versions of Rust without much
effort, and also specify what version of Rust you want in your project via the
rust-toolchain
file.
As an author, I can ensure collaborators of a project I’m working on will
download the exact version of Rust that I intended to use to develop my project.
That being said, since I’ve become more familiar with the nix package
manager, I’ve been striving to use it for all my
dependencies; this package manager is like rustup
, but for every dependency in
your system, not just the Rust toolchain. If you need a C library, it is likely
nixpkgs
has it.
The cool thing about nix as a package manager, is that you can use it the same
way you would use apt
or
brew
, but you can also specify dependencies on an specific
sub-directory, like you would do with
virtualenv
or rbenv
files. When you execute
nix-shell
on
a directory that has a shell.nix
file, boom! You are on an environment that
uses only the specified dependencies, no problems with conflicting libraries
installed on your system or anything like that.
Another great feature of nix
, is that it allows us to pin down a package
repository in a way that will make it’s packages use the same version – always.
This is similar to having your project running on a docker image or virtual
machine pinned to an specific version of ubuntu, and having the apt install
commands always resolving to the same libraries and binaries every time.
How does it do it? Well, this is not a tutorial on nix per-se, just a pitch, and a rationale about why, you, as a Rust developer, would like to try this approach to install the Rust toolchain. There are some great resources out there on how to get started with nix.
What is nixpkgs-mozilla?
It seems some folks at Mozilla also see the value of nix, and use it internally. Given this, they maintain a repository of nix packages that install the latest version of their tools and software. The mechanics about how to integrate it on a project are not trivial if you are not familiar with nix, that’s why I’m writing this blogpost.
Just to clarify, you can install Rust without Mozilla’s repository, but there is
a big chance that the version of the Rust compiler you download is not going to
be the latest stable version; also I think the Mozilla packages allow you to
cutomize the Rust toolchain more easily (target platform, stable vs nightly,
etc.) and it understands the rust-toolchain
file convention.
Implement a shell.nix
with nixpkgs-mozilla
1) Install the niv
tool to manage project deps
niv
is a tool that sits on top of nix, it
allows us to pin a package repository (“attribute set” in nix lingo) and custom
packages (“derivations” in nix lingo) using a UX friendly CLI.
Note, niv
might become obsolete in the not so distant future, the nix
community is working hard on a feature called
flakes
, which tries to
replicate what niv
does in a native way.
Is also worth mentioning, niv
is not the only way to pin dependencies, you may
be able to use nix alone for
this, however, I like niv
just
because it enforces a standard and a way to add packages (derivations) easily.
2) Initialize niv
in your directory, and add nixpkgs-mozilla
To add the nixpkgs-mozilla
package repository (“overlay” in nix lingo), we are
going to use niv add
.
$ niv init
$ niv add mozilla/nixpkgs-mozilla
Make sure to also update the nixpkgs
repository to nixpkgs-unstable
, the
nixpkgs-mozilla
overlay relies (as of Jul 2020) on code that is not in the
stable
branch.
$ niv update nixpkgs -b nixpkgs-unstable
3) Add nixpkgs-mozilla
overlay
We are going to modify the default nixpkgs “attribute set” with the Mozilla
“overlay” overrides. Now, every package that gets installed that has a Rust
dependency, will use the project’s Rust version by default. This behavior
happens because we are overriding the special rustPlatform
derivation, which
is used to build Rust programs in nix.
$ cat <<\EOF > nix/pkgs.nix
let
# import URLs for all dependencies managed by niv
sources =
import ./sources.nix;
# import the package repository (overlay) specified on nixpkgs-mozilla
mozilla-overlay =
import sources.nixpkgs-mozilla;
# define project's package repository
project-overlay =
# self and super are convoluted stuff, check out docs for nix overlays
self: super:
let
# use the utilities from the nixpkgs-mozilla to build the "rustup toolchain"
mozRustToolchain = self.rustChannelOf {
rustToolchain = ../rust-toolchain;
};
# We want to get the rust package with all these extensions
mozilla-rust = mozRustToolchain.rust.override {
extensions = [
"rust-src"
"rust-std"
"rustfmt-preview"
"rls-preview"
"clippy-preview"
];
};
# rust-src derivation is a tree of deriviations, we need to get the "src" attribute
# from one of it's paths
mozRustSrc = (builtins.elemAt mozRustToolchain.rust-src.paths 0);
# We need to modify the structure of the rust source package that comes
# from the nixpkgs-mozilla to work with an structure that works on upstream nixpkgs.
rustSrc = super.runCommandLocal "${mozRustSrc.name}-compat.tar.gz" {} ''
# get contents on directory in place
tar -xf ${mozRustSrc.src} --strip-components 1
mkdir out
# modify the directory structure to work with development/compilers/rust/rust-src.nix
mv rust-src/lib/rustlib/src/rust/* out
tar -czf rust-src.tar.gz out
# vaya con dios
mv rust-src.tar.gz $out
'';
in
{
rustPlatform = super.makeRustPlatform {
cargo = mozilla-rust;
rustc = (mozilla-rust // { src = rustSrc; });
};
mozilla-rust = mozilla-rust;
};
pinnedPkgs =
import sources.nixpkgs {
overlays = [
mozilla-overlay
project-overlay
];
};
in
pinnedPkgs
EOF
4) Now build a shell.nix
file that contains all the dependencies you need for development
$ cat <<\EOF > shell.nix
# pkgs contains all the packages from nixpkgs + the mozilla overlay
{ pkgs ? import ./nix/pkgs.nix }:
pkgs.mkShell {
buildInputs = with pkgs; [
# the package/derivation we built on nix/pkgs.nix
mozilla-rust
# other rust utilities. Because we override the rustPlatform
# package on nix/pkgs.nix, these dependencies will use the same
# version as our mozilla-rust dep.
rust-analyzer
];
}
EOF
5) Add a rust-toolchain
file
There are various valid formats for this file, to be frank, I didn’t find a place with examples, so there has been a lot of experimenting. Let us use the desired rust version we want for our project
echo '1.45.0' > rust-toolchain
6) Execute nix-shell
, and see all your dependencies get downloaded and installed
$ nix-shell
$ which rustc
/nix/store/vqqfzjjnk9zd3ps28pjxcxwzckbwlfvj-rust-1.45.0-2020-07-13-5c1f21c3b/bin/rustc
Note, this process may take longer than usual given you are compiling these
tools with the Mozilla’s Rust toolchain, and that is not going to be on the
cache server of nixpkgs
. If you want to build these packages once and share
with the rest of your team, I recommend using cachix or
roll out your own binary cache.
7) Bonus: use direnv to call nix-shell
.
This will automagically integrate a dynamic dependency environment to your editor for freeeee (if your editor supports direnv, which is likely).
Conclusion
So there you have it, it took me a while to integrate nixpkgs-mozilla
’s Rust
with the rest of the nixpkgs
ecosystem. Hopefully this will empower you to
upgrade and keep track of Rust upgrades (as well as your other deps) sanely.